2024-03-21T06:00:02.836 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-21T06:00:02.836 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi/7614674 --name yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi --verbose --owner scheduled_yingxin@teuthology --description crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} -- /home/teuthworker/archive/yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi/7614674/orig.config.yaml 2024-03-21T06:00:02.871 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T06:00:02.950 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi/7614674 branch: ci-yingxin-fix-racing-get-or-create-pg-mapping-2 description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} email: yingxin.cheng@intel.com first_in_suite: false job_id: '7614674' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: ci-yingxin-fix-racing-get-or-create-pg-mapping-2 ceph: conf: client: debug ms: 1 global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd pool default crimson: true osd: crimson alien thread cpu cores: 6-7 crimson osd obc lru size: 10 debug ms: 1 debug osd: 20 osd blocked scrub grace period: 3600 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true osd.0: crimson seastar cpu cores: 0-2 osd.1: crimson seastar cpu cores: 3-5 osd.2: crimson seastar cpu cores: 0-2 osd.3: crimson seastar cpu cores: 3-5 flavor: crimson log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 63e11116edc7c1585450815ed7030c70ea0ebef9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: crimson sha1: 63e11116edc7c1585450815ed7030c70ea0ebef9 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 workunit: branch: ci-yingxin-fix-racing-get-or-create-pg-mapping-2 env: CRIMSON_COMPAT: '1' sha1: 63e11116edc7c1585450815ed7030c70ea0ebef9 owner: scheduled_yingxin@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b sha1: 63e11116edc7c1585450815ed7030c70ea0ebef9 sleep_before_teardown: 0 suite: crimson-rados suite_branch: ci-yingxin-fix-racing-get-or-create-pg-mapping-2 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_63e11116edc7c1585450815ed7030c70ea0ebef9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 63e11116edc7c1585450815ed7030c70ea0ebef9 targets: smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIUHKbEkR+HUbnaQRi1xHhA4XWyrvOa7J+DYI/mzrXugPbvImI7V4WIeSTLo+3Bm10E3FBR8P1PMH2Py0TAZ+vs= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBPpXxYSfuuxjw/v/WXhHe553KpRC/nY5aQTyIFm3OYUyj9OKGREjr66EM1brOjwyTGoe4KJHdb0kzRgfN1wrak= tasks: - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 flavor: crimson - workunit: clients: client.0: - rados/test.sh --crimson - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-21_05:50:24 tube: smithi user: yingxin verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-21T06:00:03.022 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_63e11116edc7c1585450815ed7030c70ea0ebef9/qa; will attempt to use it 2024-03-21T06:00:03.040 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_63e11116edc7c1585450815ed7030c70ea0ebef9/qa/tasks 2024-03-21T06:00:03.041 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-21T06:00:03.044 INFO:teuthology.task.internal:Checking packages... 2024-03-21T06:00:04.427 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash '63e11116edc7c1585450815ed7030c70ea0ebef9' 2024-03-21T06:00:04.427 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-21T06:00:04.427 INFO:teuthology.packaging:ref: None 2024-03-21T06:00:04.427 INFO:teuthology.packaging:tag: None 2024-03-21T06:00:04.427 INFO:teuthology.packaging:branch: ci-yingxin-fix-racing-get-or-create-pg-mapping-2 2024-03-21T06:00:04.427 INFO:teuthology.packaging:sha1: 63e11116edc7c1585450815ed7030c70ea0ebef9 2024-03-21T06:00:04.428 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&ref=ci-yingxin-fix-racing-get-or-create-pg-mapping-2 2024-03-21T06:00:04.571 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2165.g63e11116 2024-03-21T06:00:04.573 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-21T06:00:04.581 INFO:teuthology.task.internal:no buildpackages task found 2024-03-21T06:00:04.582 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-21T06:00:04.596 INFO:teuthology.task.internal:Saving configuration 2024-03-21T06:00:04.610 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-21T06:00:04.621 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T06:00:04.645 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi/7614674', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 05:52:48.676040', 'locked_by': 'scheduled_yingxin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIUHKbEkR+HUbnaQRi1xHhA4XWyrvOa7J+DYI/mzrXugPbvImI7V4WIeSTLo+3Bm10E3FBR8P1PMH2Py0TAZ+vs='} 2024-03-21T06:00:04.664 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi/7614674', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 05:52:48.677848', 'locked_by': 'scheduled_yingxin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBPpXxYSfuuxjw/v/WXhHe553KpRC/nY5aQTyIFm3OYUyj9OKGREjr66EM1brOjwyTGoe4KJHdb0kzRgfN1wrak='} 2024-03-21T06:00:04.664 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-21T06:00:04.678 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2024-03-21T06:00:04.678 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-03-21T06:00:04.678 INFO:teuthology.run_tasks:Running task console_log... 2024-03-21T06:00:04.761 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f979409ee50>, signals=[15]) 2024-03-21T06:00:04.762 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-21T06:00:04.773 INFO:teuthology.task.internal:Opening connections... 2024-03-21T06:00:04.773 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-03-21T06:00:04.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T06:00:04.870 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2024-03-21T06:00:04.871 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T06:00:04.960 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-21T06:00:04.969 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-03-21T06:00:04.988 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-03-21T06:00:04.988 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-03-21T06:00:05.047 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2024-03-21T06:00:05.048 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="9" 2024-03-21T06:00:05.048 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2024-03-21T06:00:05.048 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2024-03-21T06:00:05.048 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="9" 2024-03-21T06:00:05.048 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el9" 2024-03-21T06:00:05.048 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T06:00:05.048 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2024-03-21T06:00:05.048 INFO:teuthology.orchestra.run.smithi096.stdout:LOGO="fedora-logo-icon" 2024-03-21T06:00:05.048 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T06:00:05.048 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2024-03-21T06:00:05.048 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T06:00:05.048 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T06:00:05.048 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T06:00:05.050 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-03-21T06:00:05.069 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2024-03-21T06:00:05.086 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2024-03-21T06:00:05.086 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2024-03-21T06:00:05.141 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream" 2024-03-21T06:00:05.153 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9" 2024-03-21T06:00:05.153 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos" 2024-03-21T06:00:05.153 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora" 2024-03-21T06:00:05.153 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9" 2024-03-21T06:00:05.153 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9" 2024-03-21T06:00:05.153 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T06:00:05.154 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2024-03-21T06:00:05.154 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon" 2024-03-21T06:00:05.154 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T06:00:05.154 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/" 2024-03-21T06:00:05.154 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T06:00:05.154 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T06:00:05.154 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T06:00:05.154 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2024-03-21T06:00:05.177 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-21T06:00:05.188 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-21T06:00:05.210 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-21T06:00:05.210 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-03-21T06:00:05.212 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2024-03-21T06:00:05.228 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-21T06:00:05.337 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-21T06:00:05.337 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-03-21T06:00:05.339 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2024-03-21T06:00:05.381 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-21T06:00:05.541 INFO:teuthology.run_tasks:Running task kernel... 2024-03-21T06:00:05.589 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-21T06:00:05.604 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-21T06:00:05.604 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T06:00:05.605 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T06:00:05.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T06:00:05.621 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-03-21T06:00:05.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T06:00:05.622 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-03-21T06:00:05.679 INFO:teuthology.orchestra.run.smithi096.stdout:5.14.0-361.el9.x86_64 2024-03-21T06:00:05.680 INFO:teuthology.task.kernel:Running kernel on smithi096: 5.14.0-361.el9.x86_64 2024-03-21T06:00:05.680 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum install -y kernel 2024-03-21T06:00:05.682 INFO:teuthology.orchestra.run.smithi184.stdout:5.14.0-361.el9.x86_64 2024-03-21T06:00:05.682 INFO:teuthology.task.kernel:Running kernel on smithi184: 5.14.0-361.el9.x86_64 2024-03-21T06:00:05.682 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum install -y kernel 2024-03-21T06:00:07.748 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-21T06:00:07.844 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-21T06:00:09.132 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2024-03-21T06:00:09.486 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-03-21T06:00:10.389 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.0 MB 00:00 2024-03-21T06:00:26.638 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-03-21T06:00:27.556 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 464 kB/s | 8.0 MB 00:17 2024-03-21T06:00:30.949 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 7.2 MB/s | 6.1 MB 00:00 2024-03-21T06:00:32.489 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-03-21T06:00:33.516 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-03-21T06:00:38.448 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-03-21T06:00:40.539 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-21T06:00:40.539 INFO:teuthology.orchestra.run.smithi184.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-21T06:00:40.617 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:00:40.631 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:00:40.631 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-03-21T06:00:40.631 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:00:40.631 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-21T06:00:40.631 INFO:teuthology.orchestra.run.smithi184.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-21T06:00:40.631 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-03-21T06:00:40.631 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-21T06:00:40.631 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-21T06:00:40.632 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-21T06:00:40.632 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:00:40.632 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-21T06:00:40.632 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:00:40.632 INFO:teuthology.orchestra.run.smithi184.stdout:Install 4 Packages 2024-03-21T06:00:40.632 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:00:40.632 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 76 M 2024-03-21T06:00:40.632 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 123 M 2024-03-21T06:00:40.632 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-21T06:00:41.192 INFO:teuthology.orchestra.run.smithi184.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 383 kB/s | 44 kB 00:00 2024-03-21T06:00:43.059 INFO:teuthology.orchestra.run.smithi184.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 7.5 MB/s | 15 MB 00:01 2024-03-21T06:00:43.851 INFO:teuthology.orchestra.run.smithi184.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 10 MB/s | 27 MB 00:02 2024-03-21T06:00:45.109 INFO:teuthology.orchestra.run.smithi184.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 8.3 MB/s | 33 MB 00:04 2024-03-21T06:00:45.109 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:00:45.109 INFO:teuthology.orchestra.run.smithi184.stdout:Total 17 MB/s | 76 MB 00:04 2024-03-21T06:00:45.778 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-21T06:00:46.137 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-21T06:00:46.137 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-21T06:00:47.151 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-21T06:00:47.152 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-21T06:00:49.622 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-21T06:00:50.255 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-21T06:00:50.343 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T06:00:53.093 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T06:00:53.247 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T06:00:55.527 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 716 kB/s | 19 MB 00:26 2024-03-21T06:00:57.920 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T06:00:57.955 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-21T06:00:59.527 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.1 MB 00:00 2024-03-21T06:01:02.203 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 10 kB/s | 16 kB 00:01 2024-03-21T06:01:02.688 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T06:01:31.744 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T06:01:31.744 INFO:teuthology.orchestra.run.smithi184.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-21T06:01:31.744 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:01:31.790 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-21T06:01:32.201 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-21T06:01:32.216 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-21T06:01:32.216 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T06:01:32.216 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T06:01:32.604 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T06:01:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:01:32.605 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-21T06:01:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-21T06:01:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-21T06:01:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-21T06:01:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-21T06:01:32.605 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:01:32.605 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:01:32.940 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q kernel | sort -rV | head -n 1 2024-03-21T06:01:33.033 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-21T06:01:33.034 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-21T06:01:33.034 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-21T06:01:33.034 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-21T06:01:33.034 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-21T06:01:33.034 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-03-21T06:01:33.034 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi184.front.sepia.ceph.com, path=None, version=distro) 2024-03-21T06:01:33.034 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q kernel | sort -rV | head -n 1 2024-03-21T06:01:33.114 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-21T06:01:33.115 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-21T06:01:33.115 DEBUG:teuthology.orchestra.run.smithi184:> sudo rpm -qi grub2-tools 2024-03-21T06:01:33.148 INFO:teuthology.orchestra.run.smithi184.stdout:Name : grub2-tools 2024-03-21T06:01:33.149 INFO:teuthology.orchestra.run.smithi184.stdout:Epoch : 1 2024-03-21T06:01:33.149 INFO:teuthology.orchestra.run.smithi184.stdout:Version : 2.06 2024-03-21T06:01:33.149 INFO:teuthology.orchestra.run.smithi184.stdout:Release : 68.el9 2024-03-21T06:01:33.149 INFO:teuthology.orchestra.run.smithi184.stdout:Architecture: x86_64 2024-03-21T06:01:33.149 INFO:teuthology.orchestra.run.smithi184.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-21T06:01:33.149 INFO:teuthology.orchestra.run.smithi184.stdout:Group : Unspecified 2024-03-21T06:01:33.149 INFO:teuthology.orchestra.run.smithi184.stdout:Size : 8264254 2024-03-21T06:01:33.149 INFO:teuthology.orchestra.run.smithi184.stdout:License : GPLv3+ 2024-03-21T06:01:33.149 INFO:teuthology.orchestra.run.smithi184.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-21T06:01:33.149 INFO:teuthology.orchestra.run.smithi184.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-21T06:01:33.149 INFO:teuthology.orchestra.run.smithi184.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-21T06:01:33.149 INFO:teuthology.orchestra.run.smithi184.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-21T06:01:33.150 INFO:teuthology.orchestra.run.smithi184.stdout:Packager : builder@centos.org 2024-03-21T06:01:33.150 INFO:teuthology.orchestra.run.smithi184.stdout:Vendor : CentOS 2024-03-21T06:01:33.150 INFO:teuthology.orchestra.run.smithi184.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-21T06:01:33.150 INFO:teuthology.orchestra.run.smithi184.stdout:Summary : Support tools for GRUB. 2024-03-21T06:01:33.150 INFO:teuthology.orchestra.run.smithi184.stdout:Description : 2024-03-21T06:01:33.150 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:01:33.150 INFO:teuthology.orchestra.run.smithi184.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-21T06:01:33.150 INFO:teuthology.orchestra.run.smithi184.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-21T06:01:33.150 INFO:teuthology.orchestra.run.smithi184.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-21T06:01:33.150 INFO:teuthology.orchestra.run.smithi184.stdout:hardware devices. 2024-03-21T06:01:33.150 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:01:33.150 INFO:teuthology.orchestra.run.smithi184.stdout:This subpackage provides tools for support of all platforms. 2024-03-21T06:01:33.151 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-21T06:01:33.151 INFO:teuthology.task.kernel:Updating grub on smithi184 to boot 5.14.0-430.el9.x86_64 2024-03-21T06:01:33.151 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-21T06:01:33.785 INFO:teuthology.orchestra.run.smithi184.stderr:Generating grub configuration file ... 2024-03-21T06:01:35.080 INFO:teuthology.orchestra.run.smithi184.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-21T06:01:35.112 INFO:teuthology.orchestra.run.smithi184.stderr:done 2024-03-21T06:01:35.114 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2024-03-21T06:01:35.166 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.pPRgsYy5Wi 2024-03-21T06:01:35.167 DEBUG:teuthology.orchestra.run.smithi184:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.pPRgsYy5Wi 2024-03-21T06:01:35.195 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.pPRgsYy5Wi 2024-03-21T06:01:35.408 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.pPRgsYy5Wi is 6KB 2024-03-21T06:01:35.455 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.pPRgsYy5Wi 2024-03-21T06:01:35.469 DEBUG:teuthology.orchestra.run.smithi184:> sudo /bin/ls /boot/loader/entries || true 2024-03-21T06:01:35.534 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-21T06:01:35.534 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-21T06:01:35.534 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-21T06:01:35.534 INFO:teuthology.orchestra.run.smithi184.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-21T06:01:35.535 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-21T06:01:35.670 DEBUG:teuthology.orchestra.run.smithi184:> sudo shutdown -r now 2024-03-21T06:02:05.674 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-03-21T06:02:05.674 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-03-21T06:02:05.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T06:02:24.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2024-03-21T06:02:33.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-03-21T06:02:33.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T06:02:36.222 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2024-03-21T06:02:48.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-03-21T06:02:48.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T06:03:20.121 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-03-21T06:03:20.523 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-03-21T06:03:20.523 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-430.el9.x86_64"... 2024-03-21T06:03:20.524 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2024-03-21T06:03:20.537 INFO:teuthology.orchestra.run.smithi184.stdout:5.14.0-430.el9.x86_64 2024-03-21T06:03:20.538 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-21T06:03:20.538 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-21T06:03:20.538 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-21T06:03:21.539 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-03-21T06:03:21.539 DEBUG:teuthology.orchestra.run.smithi184:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-21T06:03:21.707 INFO:teuthology.orchestra.run.smithi184.stdout:ttyS1 2024-03-21T06:03:21.771 DEBUG:teuthology.parallel:result is None 2024-03-21T06:03:48.631 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 127 kB/s | 21 MB 02:46 2024-03-21T06:03:53.519 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-21T06:03:55.626 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-21T06:03:55.626 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-21T06:03:55.719 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:03:55.784 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:03:55.784 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-03-21T06:03:55.784 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:03:55.784 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-21T06:03:55.784 INFO:teuthology.orchestra.run.smithi096.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-21T06:03:55.785 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-21T06:03:55.785 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-21T06:03:55.785 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-21T06:03:55.785 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-21T06:03:55.785 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:03:55.785 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-21T06:03:55.785 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:03:55.785 INFO:teuthology.orchestra.run.smithi096.stdout:Install 4 Packages 2024-03-21T06:03:55.785 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:03:55.785 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 76 M 2024-03-21T06:03:55.785 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 123 M 2024-03-21T06:03:55.785 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-21T06:03:56.121 INFO:teuthology.orchestra.run.smithi096.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 130 kB/s | 44 kB 00:00 2024-03-21T06:04:02.578 INFO:teuthology.orchestra.run.smithi096.stdout:(2/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 4.9 MB/s | 33 MB 00:06 2024-03-21T06:04:02.820 INFO:teuthology.orchestra.run.smithi096.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 4.1 MB/s | 27 MB 00:06 2024-03-21T06:04:02.970 INFO:teuthology.orchestra.run.smithi096.stdout:(4/4): kernel-core-5.14.0-430.el9.x86_64.rpm 2.1 MB/s | 15 MB 00:07 2024-03-21T06:04:02.995 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:04:02.995 INFO:teuthology.orchestra.run.smithi096.stdout:Total 10 MB/s | 76 MB 00:07 2024-03-21T06:04:03.636 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-21T06:04:03.998 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-21T06:04:03.998 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-21T06:04:05.008 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-21T06:04:05.008 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-21T06:04:07.511 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-21T06:04:08.202 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-21T06:04:08.288 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T06:04:11.086 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T06:04:11.222 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T06:04:15.950 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T06:04:16.006 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-21T06:04:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T06:04:50.270 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T06:04:50.319 INFO:teuthology.orchestra.run.smithi096.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-21T06:04:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:04:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-21T06:04:50.693 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-21T06:04:50.693 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-21T06:04:50.693 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T06:04:50.693 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T06:04:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T06:04:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:04:51.118 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-21T06:04:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-21T06:04:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-21T06:04:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-21T06:04:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-21T06:04:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:04:51.118 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:04:51.437 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q kernel | sort -rV | head -n 1 2024-03-21T06:04:51.541 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-21T06:04:51.542 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-21T06:04:51.542 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-21T06:04:51.542 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-21T06:04:51.542 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-21T06:04:51.542 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-21T06:04:51.542 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi096.front.sepia.ceph.com, path=None, version=distro) 2024-03-21T06:04:51.543 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q kernel | sort -rV | head -n 1 2024-03-21T06:04:51.622 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-21T06:04:51.622 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-21T06:04:51.641 DEBUG:teuthology.orchestra.run.smithi096:> sudo rpm -qi grub2-tools 2024-03-21T06:04:51.676 INFO:teuthology.orchestra.run.smithi096.stdout:Name : grub2-tools 2024-03-21T06:04:51.676 INFO:teuthology.orchestra.run.smithi096.stdout:Epoch : 1 2024-03-21T06:04:51.676 INFO:teuthology.orchestra.run.smithi096.stdout:Version : 2.06 2024-03-21T06:04:51.676 INFO:teuthology.orchestra.run.smithi096.stdout:Release : 68.el9 2024-03-21T06:04:51.676 INFO:teuthology.orchestra.run.smithi096.stdout:Architecture: x86_64 2024-03-21T06:04:51.676 INFO:teuthology.orchestra.run.smithi096.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-21T06:04:51.676 INFO:teuthology.orchestra.run.smithi096.stdout:Group : Unspecified 2024-03-21T06:04:51.677 INFO:teuthology.orchestra.run.smithi096.stdout:Size : 8264254 2024-03-21T06:04:51.677 INFO:teuthology.orchestra.run.smithi096.stdout:License : GPLv3+ 2024-03-21T06:04:51.677 INFO:teuthology.orchestra.run.smithi096.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-21T06:04:51.677 INFO:teuthology.orchestra.run.smithi096.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-21T06:04:51.677 INFO:teuthology.orchestra.run.smithi096.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-21T06:04:51.708 INFO:teuthology.orchestra.run.smithi096.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-21T06:04:51.708 INFO:teuthology.orchestra.run.smithi096.stdout:Packager : builder@centos.org 2024-03-21T06:04:51.709 INFO:teuthology.orchestra.run.smithi096.stdout:Vendor : CentOS 2024-03-21T06:04:51.709 INFO:teuthology.orchestra.run.smithi096.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-21T06:04:51.709 INFO:teuthology.orchestra.run.smithi096.stdout:Summary : Support tools for GRUB. 2024-03-21T06:04:51.709 INFO:teuthology.orchestra.run.smithi096.stdout:Description : 2024-03-21T06:04:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:04:51.710 INFO:teuthology.orchestra.run.smithi096.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-21T06:04:51.710 INFO:teuthology.orchestra.run.smithi096.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-21T06:04:51.710 INFO:teuthology.orchestra.run.smithi096.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-21T06:04:51.710 INFO:teuthology.orchestra.run.smithi096.stdout:hardware devices. 2024-03-21T06:04:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:04:51.710 INFO:teuthology.orchestra.run.smithi096.stdout:This subpackage provides tools for support of all platforms. 2024-03-21T06:04:51.711 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-21T06:04:51.711 INFO:teuthology.task.kernel:Updating grub on smithi096 to boot 5.14.0-430.el9.x86_64 2024-03-21T06:04:51.711 DEBUG:teuthology.orchestra.run.smithi096:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-21T06:04:52.361 INFO:teuthology.orchestra.run.smithi096.stderr:Generating grub configuration file ... 2024-03-21T06:04:53.776 INFO:teuthology.orchestra.run.smithi096.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-21T06:04:53.814 INFO:teuthology.orchestra.run.smithi096.stderr:done 2024-03-21T06:04:53.826 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-03-21T06:04:53.882 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.oRHG3IypZm 2024-03-21T06:04:53.909 DEBUG:teuthology.orchestra.run.smithi096:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.oRHG3IypZm 2024-03-21T06:04:53.953 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.oRHG3IypZm 2024-03-21T06:04:54.101 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.oRHG3IypZm is 6KB 2024-03-21T06:04:54.150 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.oRHG3IypZm 2024-03-21T06:04:54.164 DEBUG:teuthology.orchestra.run.smithi096:> sudo /bin/ls /boot/loader/entries || true 2024-03-21T06:04:54.228 INFO:teuthology.orchestra.run.smithi096.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-21T06:04:54.359 INFO:teuthology.orchestra.run.smithi096.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-21T06:04:54.359 INFO:teuthology.orchestra.run.smithi096.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-21T06:04:54.359 INFO:teuthology.orchestra.run.smithi096.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-21T06:04:54.361 DEBUG:teuthology.orchestra.run.smithi096:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-21T06:04:54.508 DEBUG:teuthology.orchestra.run.smithi096:> sudo shutdown -r now 2024-03-21T06:05:24.511 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-21T06:05:24.512 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-21T06:05:24.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T06:05:43.070 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-03-21T06:05:52.091 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-21T06:05:52.092 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T06:05:55.166 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-03-21T06:06:07.168 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-21T06:06:07.168 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T06:06:39.049 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-03-21T06:06:39.467 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-21T06:06:39.467 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-430.el9.x86_64"... 2024-03-21T06:06:39.468 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-03-21T06:06:39.483 INFO:teuthology.orchestra.run.smithi096.stdout:5.14.0-430.el9.x86_64 2024-03-21T06:06:39.484 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-21T06:06:39.484 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-21T06:06:39.484 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-21T06:06:40.484 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-21T06:06:40.485 DEBUG:teuthology.orchestra.run.smithi096:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-21T06:06:40.540 INFO:teuthology.orchestra.run.smithi096.stdout:ttyS1 2024-03-21T06:06:40.569 DEBUG:teuthology.parallel:result is None 2024-03-21T06:06:40.569 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-21T06:06:40.579 INFO:teuthology.task.internal:Creating test directory... 2024-03-21T06:06:40.579 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T06:06:40.582 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T06:06:40.600 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-21T06:06:40.606 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-21T06:06:40.613 INFO:teuthology.task.internal:Creating archive directory... 2024-03-21T06:06:40.613 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T06:06:40.635 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T06:06:40.768 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-21T06:06:40.775 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-21T06:06:40.775 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T06:06:40.777 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T06:06:40.801 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T06:06:40.814 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T06:06:40.820 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T06:06:40.834 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T06:06:40.836 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-21T06:06:40.845 INFO:teuthology.task.internal:Configuring sudo... 2024-03-21T06:06:40.845 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T06:06:40.857 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T06:06:40.956 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-21T06:06:41.025 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-21T06:06:41.026 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T06:06:41.028 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T06:06:41.044 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T06:06:41.125 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T06:06:41.183 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T06:06:41.268 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T06:06:41.313 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:06:41.313 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T06:06:41.380 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T06:06:41.426 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T06:06:41.516 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T06:06:41.557 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T06:06:41.603 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-21T06:06:41.604 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T06:06:41.674 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-03-21T06:06:41.676 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2024-03-21T06:06:41.772 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T06:06:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T06:06:42.151 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-21T06:06:42.161 INFO:teuthology.task.internal:Starting timer... 2024-03-21T06:06:42.161 INFO:teuthology.run_tasks:Running task pcp... 2024-03-21T06:06:42.286 INFO:teuthology.run_tasks:Running task selinux... 2024-03-21T06:06:42.360 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-03-21T06:06:42.360 DEBUG:teuthology.orchestra.run.smithi096:> sudo service auditd rotate 2024-03-21T06:06:42.445 INFO:teuthology.orchestra.run.smithi096.stdout:Rotating logs: 2024-03-21T06:06:42.449 DEBUG:teuthology.orchestra.run.smithi184:> sudo service auditd rotate 2024-03-21T06:06:42.506 INFO:teuthology.orchestra.run.smithi184.stdout:Rotating logs: 2024-03-21T06:06:42.507 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-21T06:06:42.507 DEBUG:teuthology.orchestra.run.smithi096:> /usr/sbin/getenforce 2024-03-21T06:06:42.541 INFO:teuthology.orchestra.run.smithi096.stdout:Permissive 2024-03-21T06:06:42.541 DEBUG:teuthology.orchestra.run.smithi184:> /usr/sbin/getenforce 2024-03-21T06:06:42.561 INFO:teuthology.orchestra.run.smithi184.stdout:Permissive 2024-03-21T06:06:42.561 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi096.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi184.front.sepia.ceph.com': 'permissive'} 2024-03-21T06:06:42.561 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T06:06:42.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T06:06:42.646 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T06:06:42.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T06:06:42.674 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-21T06:06:42.675 DEBUG:teuthology.orchestra.run.smithi096:> sudo /usr/sbin/setenforce permissive 2024-03-21T06:06:42.942 DEBUG:teuthology.orchestra.run.smithi184:> sudo /usr/sbin/setenforce permissive 2024-03-21T06:06:42.979 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-21T06:06:43.340 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-21T06:06:43.423 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-21T06:06:46.447 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-21T06:06:49.535 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-21T06:06:49.538 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi096.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-21T06:12:01.492 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2024-03-21T06:12:01.541 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-21T06:12:01.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T06:12:01.636 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-03-21T06:12:01.719 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-21T06:12:01.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-03-21T06:12:01.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T06:12:01.809 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-03-21T06:12:01.890 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-03-21T06:12:01.890 INFO:teuthology.run_tasks:Running task clock... 2024-03-21T06:12:01.960 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-21T06:12:02.028 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T06:12:02.028 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T06:12:02.031 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T06:12:02.031 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T06:12:02.064 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T06:12:02.066 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T06:12:02.082 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T06:12:02.084 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T06:12:02.113 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: ntpd: command not found 2024-03-21T06:12:02.114 INFO:teuthology.orchestra.run.smithi184.stderr:sudo: ntpd: command not found 2024-03-21T06:12:02.127 INFO:teuthology.orchestra.run.smithi096.stdout:506 Cannot talk to daemon 2024-03-21T06:12:02.128 INFO:teuthology.orchestra.run.smithi184.stdout:506 Cannot talk to daemon 2024-03-21T06:12:02.144 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T06:12:02.145 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T06:12:02.163 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T06:12:02.164 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T06:12:02.212 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-03-21T06:12:02.214 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-03-21T06:12:02.215 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T06:12:02.215 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-03-21T06:12:02.215 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T06:12:02.215 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T06:12:02.215 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T06:12:02.216 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T06:12:02.217 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T06:12:02.217 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-03-21T06:12:02.218 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T06:12:02.218 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T06:12:02.218 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T06:12:02.218 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T06:12:02.219 INFO:teuthology.run_tasks:Running task install... 2024-03-21T06:12:02.228 DEBUG:teuthology.task.install:project ceph 2024-03-21T06:12:02.228 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': '63e11116edc7c1585450815ed7030c70ea0ebef9'}} 2024-03-21T06:12:02.228 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': '63e11116edc7c1585450815ed7030c70ea0ebef9'} 2024-03-21T06:12:02.228 INFO:teuthology.task.install:Using flavor: crimson 2024-03-21T06:12:02.252 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-21T06:12:02.252 INFO:teuthology.task.install:extra packages: [] 2024-03-21T06:12:02.252 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '63e11116edc7c1585450815ed7030c70ea0ebef9', 'tag': None, 'wait_for_package': False} 2024-03-21T06:12:02.253 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=63e11116edc7c1585450815ed7030c70ea0ebef9 2024-03-21T06:12:02.255 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '63e11116edc7c1585450815ed7030c70ea0ebef9', 'tag': None, 'wait_for_package': False} 2024-03-21T06:12:02.255 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=63e11116edc7c1585450815ed7030c70ea0ebef9 2024-03-21T06:12:02.394 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/ci-yingxin-fix-racing-get-or-create-pg-mapping-2/63e11116edc7c1585450815ed7030c70ea0ebef9/centos/9/flavors/crimson/ 2024-03-21T06:12:02.395 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2165.g63e11116 2024-03-21T06:12:02.412 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/ci-yingxin-fix-racing-get-or-create-pg-mapping-2/63e11116edc7c1585450815ed7030c70ea0ebef9/centos/9/flavors/crimson/ 2024-03-21T06:12:02.412 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2165.g63e11116 2024-03-21T06:12:02.754 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/ci-yingxin-fix-racing-get-or-create-pg-mapping-2/63e11116edc7c1585450815ed7030c70ea0ebef9/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/ci-yingxin-fix-racing-get-or-create-pg-mapping-2/63e11116edc7c1585450815ed7030c70ea0ebef9/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/ci-yingxin-fix-racing-get-or-create-pg-mapping-2/63e11116edc7c1585450815ed7030c70ea0ebef9/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T06:12:02.754 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-21T06:12:02.754 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T06:12:02.755 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/ci-yingxin-fix-racing-get-or-create-pg-mapping-2/63e11116edc7c1585450815ed7030c70ea0ebef9/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/ci-yingxin-fix-racing-get-or-create-pg-mapping-2/63e11116edc7c1585450815ed7030c70ea0ebef9/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/ci-yingxin-fix-racing-get-or-create-pg-mapping-2/63e11116edc7c1585450815ed7030c70ea0ebef9/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T06:12:02.755 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:12:02.755 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T06:12:02.783 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-21T06:12:02.783 DEBUG:teuthology.orchestra.run.smithi184:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/63e11116edc7c1585450815ed7030c70ea0ebef9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T06:12:02.784 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-21T06:12:02.784 DEBUG:teuthology.orchestra.run.smithi096:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/63e11116edc7c1585450815ed7030c70ea0ebef9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T06:12:02.851 DEBUG:teuthology.orchestra.run.smithi184:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T06:12:02.852 DEBUG:teuthology.orchestra.run.smithi096:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T06:12:02.931 DEBUG:teuthology.orchestra.run.smithi184:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T06:12:02.932 DEBUG:teuthology.orchestra.run.smithi096:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T06:12:02.997 INFO:teuthology.orchestra.run.smithi184.stdout:check_obsoletes = 1 2024-03-21T06:12:02.998 INFO:teuthology.orchestra.run.smithi096.stdout:check_obsoletes = 1 2024-03-21T06:12:02.999 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum clean all 2024-03-21T06:12:03.000 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum clean all 2024-03-21T06:12:03.238 INFO:teuthology.orchestra.run.smithi096.stdout:74 files removed 2024-03-21T06:12:03.247 INFO:teuthology.orchestra.run.smithi184.stdout:74 files removed 2024-03-21T06:12:03.265 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-radosgw 2024-03-21T06:12:03.273 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-radosgw 2024-03-21T06:12:04.300 INFO:teuthology.orchestra.run.smithi096.stdout:ceph packages for x86_64 117 kB/s | 86 kB 00:00 2024-03-21T06:12:04.315 INFO:teuthology.orchestra.run.smithi184.stdout:ceph packages for x86_64 117 kB/s | 86 kB 00:00 2024-03-21T06:12:05.001 INFO:teuthology.orchestra.run.smithi096.stdout:ceph noarch packages 15 kB/s | 10 kB 00:00 2024-03-21T06:12:05.023 INFO:teuthology.orchestra.run.smithi184.stdout:ceph noarch packages 15 kB/s | 10 kB 00:00 2024-03-21T06:12:05.734 INFO:teuthology.orchestra.run.smithi096.stdout:ceph source packages 2.9 kB/s | 2.0 kB 00:00 2024-03-21T06:12:05.774 INFO:teuthology.orchestra.run.smithi184.stdout:ceph source packages 2.8 kB/s | 2.0 kB 00:00 2024-03-21T06:12:06.218 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-21T06:12:06.412 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-03-21T06:12:07.903 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2024-03-21T06:12:07.943 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS-9 - BaseOS 3.3 MB/s | 2.0 MB 00:00 2024-03-21T06:12:11.958 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.0 MB 00:03 2024-03-21T06:12:14.437 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-03-21T06:12:15.051 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 1.2 MB/s | 8.0 MB 00:06 2024-03-21T06:12:21.485 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 3.5 MB/s | 19 MB 00:05 2024-03-21T06:12:25.702 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 807 kB/s | 6.1 MB 00:07 2024-03-21T06:12:27.518 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-03-21T06:12:27.826 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.1 MB 00:02 2024-03-21T06:12:28.503 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-03-21T06:12:29.402 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 75 kB/s | 16 kB 00:00 2024-03-21T06:12:30.395 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-03-21T06:12:33.386 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-03-21T06:12:35.270 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-03-21T06:12:35.807 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:12:35.820 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:12:35.821 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-03-21T06:12:35.821 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:12:35.821 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-21T06:12:35.821 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw x86_64 2:19.0.0-2165.g63e11116.el9 ceph 9.5 M 2024-03-21T06:12:35.821 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2024-03-21T06:12:35.821 INFO:teuthology.orchestra.run.smithi184.stdout: librados2 x86_64 2:19.0.0-2165.g63e11116.el9 ceph 3.2 M 2024-03-21T06:12:35.821 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1 x86_64 2:19.0.0-2165.g63e11116.el9 ceph 3.1 M 2024-03-21T06:12:35.821 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-03-21T06:12:35.821 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options 2024-03-21T06:12:35.821 INFO:teuthology.orchestra.run.smithi184.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T06:12:35.821 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base x86_64 2:19.0.0-2165.g63e11116.el9 ceph 5.4 M 2024-03-21T06:12:35.821 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common x86_64 2:19.0.0-2165.g63e11116.el9 ceph 19 M 2024-03-21T06:12:35.822 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux x86_64 2:19.0.0-2165.g63e11116.el9 ceph 25 k 2024-03-21T06:12:35.822 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T06:12:35.822 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T06:12:35.822 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T06:12:35.822 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2 x86_64 2:19.0.0-2165.g63e11116.el9 ceph 754 k 2024-03-21T06:12:35.822 INFO:teuthology.orchestra.run.smithi184.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T06:12:35.822 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T06:12:35.822 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T06:12:35.822 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 x86_64 2:19.0.0-2165.g63e11116.el9 ceph 496 k 2024-03-21T06:12:35.822 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T06:12:35.822 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2 x86_64 2:19.0.0-2165.g63e11116.el9 ceph 4.9 M 2024-03-21T06:12:35.822 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T06:12:35.822 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T06:12:35.823 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T06:12:35.823 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse 2024-03-21T06:12:35.823 INFO:teuthology.orchestra.run.smithi184.stdout: x86_64 2:19.0.0-2165.g63e11116.el9 ceph 45 k 2024-03-21T06:12:35.823 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common x86_64 2:19.0.0-2165.g63e11116.el9 ceph 131 k 2024-03-21T06:12:35.823 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs x86_64 2:19.0.0-2165.g63e11116.el9 ceph 162 k 2024-03-21T06:12:35.823 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T06:12:35.823 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados x86_64 2:19.0.0-2165.g63e11116.el9 ceph 320 k 2024-03-21T06:12:35.823 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd x86_64 2:19.0.0-2165.g63e11116.el9 ceph 299 k 2024-03-21T06:12:35.823 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw x86_64 2:19.0.0-2165.g63e11116.el9 ceph 99 k 2024-03-21T06:12:35.823 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T06:12:35.823 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T06:12:35.823 INFO:teuthology.orchestra.run.smithi184.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T06:12:35.824 INFO:teuthology.orchestra.run.smithi184.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T06:12:35.824 INFO:teuthology.orchestra.run.smithi184.stdout:Installing weak dependencies: 2024-03-21T06:12:35.824 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T06:12:35.824 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:12:35.824 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-21T06:12:35.824 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:12:35.824 INFO:teuthology.orchestra.run.smithi184.stdout:Install 30 Packages 2024-03-21T06:12:35.824 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 2 Packages 2024-03-21T06:12:35.824 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:12:35.825 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 56 M 2024-03-21T06:12:35.825 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-21T06:12:37.684 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:12:37.687 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:12:37.687 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-21T06:12:37.687 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:12:37.687 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-21T06:12:37.688 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw x86_64 2:19.0.0-2165.g63e11116.el9 ceph 9.5 M 2024-03-21T06:12:37.688 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrading: 2024-03-21T06:12:37.688 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 x86_64 2:19.0.0-2165.g63e11116.el9 ceph 3.2 M 2024-03-21T06:12:37.688 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1 x86_64 2:19.0.0-2165.g63e11116.el9 ceph 3.1 M 2024-03-21T06:12:37.688 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-21T06:12:37.688 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options 2024-03-21T06:12:37.688 INFO:teuthology.orchestra.run.smithi096.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T06:12:37.688 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base x86_64 2:19.0.0-2165.g63e11116.el9 ceph 5.4 M 2024-03-21T06:12:37.688 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common x86_64 2:19.0.0-2165.g63e11116.el9 ceph 19 M 2024-03-21T06:12:37.688 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux x86_64 2:19.0.0-2165.g63e11116.el9 ceph 25 k 2024-03-21T06:12:37.688 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T06:12:37.688 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T06:12:37.688 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T06:12:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2 x86_64 2:19.0.0-2165.g63e11116.el9 ceph 754 k 2024-03-21T06:12:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T06:12:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T06:12:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T06:12:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 x86_64 2:19.0.0-2165.g63e11116.el9 ceph 496 k 2024-03-21T06:12:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T06:12:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2 x86_64 2:19.0.0-2165.g63e11116.el9 ceph 4.9 M 2024-03-21T06:12:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T06:12:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T06:12:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T06:12:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse 2024-03-21T06:12:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: x86_64 2:19.0.0-2165.g63e11116.el9 ceph 45 k 2024-03-21T06:12:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common x86_64 2:19.0.0-2165.g63e11116.el9 ceph 131 k 2024-03-21T06:12:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs x86_64 2:19.0.0-2165.g63e11116.el9 ceph 162 k 2024-03-21T06:12:37.690 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T06:12:37.690 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados x86_64 2:19.0.0-2165.g63e11116.el9 ceph 320 k 2024-03-21T06:12:37.690 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd x86_64 2:19.0.0-2165.g63e11116.el9 ceph 299 k 2024-03-21T06:12:37.690 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw x86_64 2:19.0.0-2165.g63e11116.el9 ceph 99 k 2024-03-21T06:12:37.690 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T06:12:37.690 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T06:12:37.690 INFO:teuthology.orchestra.run.smithi096.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T06:12:37.690 INFO:teuthology.orchestra.run.smithi096.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T06:12:37.690 INFO:teuthology.orchestra.run.smithi096.stdout:Installing weak dependencies: 2024-03-21T06:12:37.690 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T06:12:37.690 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:12:37.690 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-21T06:12:37.690 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:12:37.690 INFO:teuthology.orchestra.run.smithi096.stdout:Install 30 Packages 2024-03-21T06:12:37.691 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrade 2 Packages 2024-03-21T06:12:37.691 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:12:37.691 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 56 M 2024-03-21T06:12:37.691 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-21T06:12:41.054 INFO:teuthology.orchestra.run.smithi184.stdout:(1/32): ceph-radosgw-19.0.0-2165.g63e11116.el9. 2.0 MB/s | 9.5 MB 00:04 2024-03-21T06:12:41.162 INFO:teuthology.orchestra.run.smithi184.stdout:(2/32): ceph-selinux-19.0.0-2165.g63e11116.el9. 233 kB/s | 25 kB 00:00 2024-03-21T06:12:41.546 INFO:teuthology.orchestra.run.smithi184.stdout:(3/32): ceph-base-19.0.0-2165.g63e11116.el9.x86 1.0 MB/s | 5.4 MB 00:05 2024-03-21T06:12:41.854 INFO:teuthology.orchestra.run.smithi184.stdout:(4/32): libradosstriper1-19.0.0-2165.g63e11116. 1.6 MB/s | 496 kB 00:00 2024-03-21T06:12:41.980 INFO:teuthology.orchestra.run.smithi184.stdout:(5/32): libcephfs2-19.0.0-2165.g63e11116.el9.x8 923 kB/s | 754 kB 00:00 2024-03-21T06:12:42.072 INFO:teuthology.orchestra.run.smithi184.stdout:(6/32): python3-ceph-argparse-19.0.0-2165.g63e1 490 kB/s | 45 kB 00:00 2024-03-21T06:12:42.255 INFO:teuthology.orchestra.run.smithi184.stdout:(7/32): python3-ceph-common-19.0.0-2165.g63e111 716 kB/s | 131 kB 00:00 2024-03-21T06:12:42.397 INFO:teuthology.orchestra.run.smithi184.stdout:(8/32): python3-cephfs-19.0.0-2165.g63e11116.el 1.1 MB/s | 162 kB 00:00 2024-03-21T06:12:42.589 INFO:teuthology.orchestra.run.smithi184.stdout:(9/32): python3-rados-19.0.0-2165.g63e11116.el9 1.6 MB/s | 320 kB 00:00 2024-03-21T06:12:42.781 INFO:teuthology.orchestra.run.smithi184.stdout:(10/32): python3-rbd-19.0.0-2165.g63e11116.el9. 1.5 MB/s | 299 kB 00:00 2024-03-21T06:12:42.865 INFO:teuthology.orchestra.run.smithi184.stdout:(11/32): python3-rgw-19.0.0-2165.g63e11116.el9. 1.2 MB/s | 99 kB 00:00 2024-03-21T06:12:42.973 INFO:teuthology.orchestra.run.smithi184.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 977 kB/s | 106 kB 00:00 2024-03-21T06:12:43.024 INFO:teuthology.orchestra.run.smithi184.stdout:(13/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-21T06:12:43.057 INFO:teuthology.orchestra.run.smithi184.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-21T06:12:43.141 INFO:teuthology.orchestra.run.smithi184.stdout:(15/32): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 661 kB 00:00 2024-03-21T06:12:43.191 INFO:teuthology.orchestra.run.smithi184.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-21T06:12:43.235 INFO:teuthology.orchestra.run.smithi184.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 994 kB/s | 41 kB 00:00 2024-03-21T06:12:43.645 INFO:teuthology.orchestra.run.smithi096.stdout:(1/32): ceph-radosgw-19.0.0-2165.g63e11116.el9. 1.7 MB/s | 9.5 MB 00:05 2024-03-21T06:12:43.720 INFO:teuthology.orchestra.run.smithi096.stdout:(2/32): ceph-selinux-19.0.0-2165.g63e11116.el9. 337 kB/s | 25 kB 00:00 2024-03-21T06:12:44.158 INFO:teuthology.orchestra.run.smithi184.stdout:(18/32): qatlib-23.11.0-1.el9.x86_64.rpm 234 kB/s | 216 kB 00:00 2024-03-21T06:12:44.178 INFO:teuthology.orchestra.run.smithi096.stdout:(3/32): libcephfs2-19.0.0-2165.g63e11116.el9.x8 1.6 MB/s | 754 kB 00:00 2024-03-21T06:12:44.400 INFO:teuthology.orchestra.run.smithi184.stdout:(19/32): librgw2-19.0.0-2165.g63e11116.el9.x86_ 1.9 MB/s | 4.9 MB 00:02 2024-03-21T06:12:44.529 INFO:teuthology.orchestra.run.smithi096.stdout:(4/32): libradosstriper1-19.0.0-2165.g63e11116. 1.4 MB/s | 496 kB 00:00 2024-03-21T06:12:44.792 INFO:teuthology.orchestra.run.smithi184.stdout:(20/32): qatlib-service-23.11.0-1.el9.x86_64.rp 55 kB/s | 35 kB 00:00 2024-03-21T06:12:45.051 INFO:teuthology.orchestra.run.smithi184.stdout:(21/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 71 kB/s | 44 kB 00:00 2024-03-21T06:12:45.326 INFO:teuthology.orchestra.run.smithi184.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 579 kB/s | 309 kB 00:00 2024-03-21T06:12:45.435 INFO:teuthology.orchestra.run.smithi184.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-03-21T06:12:45.560 INFO:teuthology.orchestra.run.smithi184.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 388 kB/s | 49 kB 00:00 2024-03-21T06:12:45.694 INFO:teuthology.orchestra.run.smithi184.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 505 kB/s | 67 kB 00:00 2024-03-21T06:12:45.862 INFO:teuthology.orchestra.run.smithi096.stdout:(5/32): ceph-base-19.0.0-2165.g63e11116.el9.x86 713 kB/s | 5.4 MB 00:07 2024-03-21T06:12:45.954 INFO:teuthology.orchestra.run.smithi096.stdout:(6/32): python3-ceph-argparse-19.0.0-2165.g63e1 494 kB/s | 45 kB 00:00 2024-03-21T06:12:46.019 INFO:teuthology.orchestra.run.smithi184.stdout:(26/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.5 MB/s | 838 kB 00:00 2024-03-21T06:12:46.161 INFO:teuthology.orchestra.run.smithi184.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-03-21T06:12:46.219 INFO:teuthology.orchestra.run.smithi184.stdout:(28/32): libarrow-9.0.0-10.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-03-21T06:12:46.254 INFO:teuthology.orchestra.run.smithi096.stdout:(7/32): python3-ceph-common-19.0.0-2165.g63e111 437 kB/s | 131 kB 00:00 2024-03-21T06:12:46.403 INFO:teuthology.orchestra.run.smithi184.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 6.5 MB/s | 1.6 MB 00:00 2024-03-21T06:12:46.446 INFO:teuthology.orchestra.run.smithi096.stdout:(8/32): python3-cephfs-19.0.0-2165.g63e11116.el 846 kB/s | 162 kB 00:00 2024-03-21T06:12:46.805 INFO:teuthology.orchestra.run.smithi096.stdout:(9/32): python3-rados-19.0.0-2165.g63e11116.el9 894 kB/s | 320 kB 00:00 2024-03-21T06:12:46.828 INFO:teuthology.orchestra.run.smithi184.stdout:(30/32): ceph-common-19.0.0-2165.g63e11116.el9. 1.8 MB/s | 19 MB 00:10 2024-03-21T06:12:46.921 INFO:teuthology.orchestra.run.smithi096.stdout:(10/32): librgw2-19.0.0-2165.g63e11116.el9.x86_ 2.0 MB/s | 4.9 MB 00:02 2024-03-21T06:12:47.005 INFO:teuthology.orchestra.run.smithi096.stdout:(11/32): python3-rgw-19.0.0-2165.g63e11116.el9. 1.2 MB/s | 99 kB 00:00 2024-03-21T06:12:47.122 INFO:teuthology.orchestra.run.smithi096.stdout:(12/32): python3-rbd-19.0.0-2165.g63e11116.el9. 943 kB/s | 299 kB 00:00 2024-03-21T06:12:47.139 INFO:teuthology.orchestra.run.smithi096.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 791 kB/s | 106 kB 00:00 2024-03-21T06:12:47.165 INFO:teuthology.orchestra.run.smithi096.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-21T06:12:47.190 INFO:teuthology.orchestra.run.smithi096.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 881 kB/s | 44 kB 00:00 2024-03-21T06:12:47.223 INFO:teuthology.orchestra.run.smithi096.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-21T06:12:47.249 INFO:teuthology.orchestra.run.smithi096.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-03-21T06:12:47.274 INFO:teuthology.orchestra.run.smithi096.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 831 kB/s | 41 kB 00:00 2024-03-21T06:12:47.399 INFO:teuthology.orchestra.run.smithi096.stdout:(19/32): qatlib-service-23.11.0-1.el9.x86_64.rp 279 kB/s | 35 kB 00:00 2024-03-21T06:12:47.508 INFO:teuthology.orchestra.run.smithi096.stdout:(20/32): qatlib-23.11.0-1.el9.x86_64.rpm 837 kB/s | 216 kB 00:00 2024-03-21T06:12:47.612 INFO:teuthology.orchestra.run.smithi184.stdout:(31/32): librados2-19.0.0-2165.g63e11116.el9.x8 2.3 MB/s | 3.2 MB 00:01 2024-03-21T06:12:47.845 INFO:teuthology.orchestra.run.smithi184.stdout:(32/32): librbd1-19.0.0-2165.g63e11116.el9.x86_ 2.1 MB/s | 3.1 MB 00:01 2024-03-21T06:12:47.849 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:12:47.849 INFO:teuthology.orchestra.run.smithi184.stdout:Total 4.7 MB/s | 56 MB 00:12 2024-03-21T06:12:47.884 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-21T06:12:47.922 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-21T06:12:47.922 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-21T06:12:48.382 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-21T06:12:48.383 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-21T06:12:48.899 INFO:teuthology.orchestra.run.smithi096.stdout:(21/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 30 kB/s | 44 kB 00:01 2024-03-21T06:12:49.826 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-21T06:12:50.229 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-03-21T06:12:50.264 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librados2-2:19.0.0-2165.g63e11116.el9.x86_64 2/34 2024-03-21T06:12:50.355 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:19.0.0-2165.g63e11116.el9.x86_64 2/34 2024-03-21T06:12:50.383 INFO:teuthology.orchestra.run.smithi096.stdout:(22/32): ceph-common-19.0.0-2165.g63e11116.el9. 1.5 MB/s | 19 MB 00:12 2024-03-21T06:12:50.454 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rados-2:19.0.0-2165.g63e11116.el9.x86_64 3/34 2024-03-21T06:12:50.482 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs2-2:19.0.0-2165.g63e11116.el9.x86_64 4/34 2024-03-21T06:12:50.500 INFO:teuthology.orchestra.run.smithi096.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 223 kB/s | 26 kB 00:00 2024-03-21T06:12:50.562 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs2-2:19.0.0-2165.g63e11116.el9.x86_64 4/34 2024-03-21T06:12:50.592 INFO:teuthology.orchestra.run.smithi096.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 530 kB/s | 49 kB 00:00 2024-03-21T06:12:50.594 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/34 2024-03-21T06:12:50.620 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/34 2024-03-21T06:12:50.643 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 7/34 2024-03-21T06:12:50.650 INFO:teuthology.orchestra.run.smithi096.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-21T06:12:50.659 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 7/34 2024-03-21T06:12:50.804 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 8/34 2024-03-21T06:12:50.884 INFO:teuthology.orchestra.run.smithi096.stdout:(26/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.5 MB/s | 838 kB 00:00 2024-03-21T06:12:50.915 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 8/34 2024-03-21T06:12:51.003 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/34 2024-03-21T06:12:51.047 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-argparse-2:19.0.0-2165.g63e11116.el9. 10/34 2024-03-21T06:12:51.074 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cephfs-2:19.0.0-2165.g63e11116.el9.x86_64 11/34 2024-03-21T06:12:51.143 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 12/34 2024-03-21T06:12:51.159 INFO:teuthology.orchestra.run.smithi096.stdout:(27/32): libarrow-9.0.0-10.el9.x86_64.rpm 2.0 MB/s | 4.4 MB 00:02 2024-03-21T06:12:51.164 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libradosstriper1-2:19.0.0-2165.g63e11116.el9.x86_6 13/34 2024-03-21T06:12:51.253 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2165.g63e11116.el9.x86_6 13/34 2024-03-21T06:12:51.289 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : thrift-0.15.0-2.el9.x86_64 14/34 2024-03-21T06:12:51.400 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-03-21T06:12:51.426 INFO:teuthology.orchestra.run.smithi096.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 354 kB/s | 192 kB 00:00 2024-03-21T06:12:51.445 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-03-21T06:12:51.489 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-03-21T06:12:51.529 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-03-21T06:12:51.559 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-03-21T06:12:51.591 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-03-21T06:12:51.940 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-03-21T06:12:51.971 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librbd1-2:19.0.0-2165.g63e11116.el9.x86_64 22/34 2024-03-21T06:12:52.062 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:19.0.0-2165.g63e11116.el9.x86_64 22/34 2024-03-21T06:12:52.093 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rbd-2:19.0.0-2165.g63e11116.el9.x86_64 23/34 2024-03-21T06:12:52.138 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-03-21T06:12:52.617 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-03-21T06:12:52.648 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librgw2-2:19.0.0-2165.g63e11116.el9.x86_64 26/34 2024-03-21T06:12:52.704 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librgw2-2:19.0.0-2165.g63e11116.el9.x86_64 26/34 2024-03-21T06:12:52.783 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rgw-2:19.0.0-2165.g63e11116.el9.x86_64 27/34 2024-03-21T06:12:52.809 INFO:teuthology.orchestra.run.smithi096.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 982 kB/s | 1.6 MB 00:01 2024-03-21T06:12:52.810 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-common-2:19.0.0-2165.g63e11116.el9.x8 28/34 2024-03-21T06:12:54.567 INFO:teuthology.orchestra.run.smithi096.stdout:(30/32): librados2-19.0.0-2165.g63e11116.el9.x8 1.0 MB/s | 3.2 MB 00:03 2024-03-21T06:12:54.693 INFO:teuthology.orchestra.run.smithi096.stdout:(31/32): librbd1-19.0.0-2165.g63e11116.el9.x86_ 1.6 MB/s | 3.1 MB 00:01 2024-03-21T06:12:54.909 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.0.0-2165.g63e11116.el9.x86_64 29/34 2024-03-21T06:12:54.947 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-common-2:19.0.0-2165.g63e11116.el9.x86_64 29/34 2024-03-21T06:12:55.521 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.0.0-2165.g63e11116.el9.x86_64 29/34 2024-03-21T06:12:55.559 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-base-2:19.0.0-2165.g63e11116.el9.x86_64 30/34 2024-03-21T06:12:55.650 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-base-2:19.0.0-2165.g63e11116.el9.x86_64 30/34 2024-03-21T06:12:55.650 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T06:12:55.650 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T06:12:55.650 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:12:55.694 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-selinux-2:19.0.0-2165.g63e11116.el9.x86_64 31/34 2024-03-21T06:13:04.988 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2165.g63e11116.el9.x86_64 31/34 2024-03-21T06:13:04.989 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /sys 2024-03-21T06:13:04.989 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /proc 2024-03-21T06:13:04.989 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /mnt 2024-03-21T06:13:04.989 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /var/tmp 2024-03-21T06:13:04.989 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /home 2024-03-21T06:13:04.989 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /root 2024-03-21T06:13:04.989 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /tmp 2024-03-21T06:13:04.989 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:13:05.919 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-radosgw-2:19.0.0-2165.g63e11116.el9.x86_64 32/34 2024-03-21T06:13:05.944 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2165.g63e11116.el9.x86_64 32/34 2024-03-21T06:13:05.944 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T06:13:05.944 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T06:13:05.944 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T06:13:05.944 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T06:13:05.945 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:13:05.946 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-21T06:13:06.029 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-21T06:13:06.029 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T06:13:07.224 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T06:13:07.224 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-base-2:19.0.0-2165.g63e11116.el9.x86_64 1/34 2024-03-21T06:13:07.224 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-common-2:19.0.0-2165.g63e11116.el9.x86_64 2/34 2024-03-21T06:13:07.224 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-radosgw-2:19.0.0-2165.g63e11116.el9.x86_64 3/34 2024-03-21T06:13:07.224 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-selinux-2:19.0.0-2165.g63e11116.el9.x86_64 4/34 2024-03-21T06:13:07.224 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs2-2:19.0.0-2165.g63e11116.el9.x86_64 5/34 2024-03-21T06:13:07.224 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libradosstriper1-2:19.0.0-2165.g63e11116.el9.x86_6 6/34 2024-03-21T06:13:07.224 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librgw2-2:19.0.0-2165.g63e11116.el9.x86_64 7/34 2024-03-21T06:13:07.224 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2165.g63e11116.el9. 8/34 2024-03-21T06:13:07.225 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-common-2:19.0.0-2165.g63e11116.el9.x8 9/34 2024-03-21T06:13:07.225 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cephfs-2:19.0.0-2165.g63e11116.el9.x86_64 10/34 2024-03-21T06:13:07.225 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rados-2:19.0.0-2165.g63e11116.el9.x86_64 11/34 2024-03-21T06:13:07.225 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rbd-2:19.0.0-2165.g63e11116.el9.x86_64 12/34 2024-03-21T06:13:07.225 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rgw-2:19.0.0-2165.g63e11116.el9.x86_64 13/34 2024-03-21T06:13:07.225 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-03-21T06:13:07.225 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-03-21T06:13:07.225 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-03-21T06:13:07.225 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-03-21T06:13:07.225 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-03-21T06:13:07.225 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-03-21T06:13:07.225 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-03-21T06:13:07.225 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-03-21T06:13:07.226 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-03-21T06:13:07.226 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-03-21T06:13:07.226 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-03-21T06:13:07.226 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-03-21T06:13:07.226 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-03-21T06:13:07.226 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-03-21T06:13:07.226 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-03-21T06:13:07.226 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-03-21T06:13:07.227 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-03-21T06:13:07.227 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:19.0.0-2165.g63e11116.el9.x86_64 31/34 2024-03-21T06:13:07.227 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-03-21T06:13:07.227 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:19.0.0-2165.g63e11116.el9.x86_64 33/34 2024-03-21T06:13:08.261 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T06:13:08.261 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:13:08.261 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2024-03-21T06:13:08.261 INFO:teuthology.orchestra.run.smithi184.stdout: librados2-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:13:08.262 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:13:08.262 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-21T06:13:08.262 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T06:13:08.262 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:13:08.262 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:13:08.262 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:13:08.262 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:13:08.262 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T06:13:08.262 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T06:13:08.262 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T06:13:08.262 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:13:08.262 INFO:teuthology.orchestra.run.smithi184.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T06:13:08.262 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T06:13:08.263 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T06:13:08.263 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:13:08.263 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T06:13:08.263 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:13:08.263 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T06:13:08.263 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T06:13:08.263 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T06:13:08.263 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:13:08.263 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:13:08.263 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:13:08.263 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T06:13:08.263 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:13:08.263 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:13:08.263 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:13:08.263 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T06:13:08.263 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T06:13:08.264 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T06:13:08.264 INFO:teuthology.orchestra.run.smithi184.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T06:13:08.264 INFO:teuthology.orchestra.run.smithi184.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T06:13:08.264 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:13:08.264 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:13:08.555 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-test 2024-03-21T06:13:09.108 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:00:36 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:13:09.265 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:13:09.266 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:13:09.266 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-03-21T06:13:09.266 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:13:09.266 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-21T06:13:09.266 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test x86_64 2:19.0.0-2165.g63e11116.el9 ceph 123 M 2024-03-21T06:13:09.267 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-03-21T06:13:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T06:13:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: libasan x86_64 11.4.1-3.el9 appstream 418 k 2024-03-21T06:13:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite x86_64 2:19.0.0-2165.g63e11116.el9 ceph 158 k 2024-03-21T06:13:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: libubsan x86_64 11.4.1-3.el9 appstream 187 k 2024-03-21T06:13:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T06:13:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T06:13:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T06:13:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-03-21T06:13:09.267 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:13:09.267 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-21T06:13:09.267 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:13:09.268 INFO:teuthology.orchestra.run.smithi184.stdout:Install 9 Packages 2024-03-21T06:13:09.268 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:13:09.268 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 124 M 2024-03-21T06:13:09.268 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 1.1 G 2024-03-21T06:13:09.268 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-21T06:13:09.582 INFO:teuthology.orchestra.run.smithi184.stdout:(1/9): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.3 MB/s | 217 kB 00:00 2024-03-21T06:13:09.624 INFO:teuthology.orchestra.run.smithi184.stdout:(2/9): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 300 kB 00:00 2024-03-21T06:13:09.657 INFO:teuthology.orchestra.run.smithi184.stdout:(3/9): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-21T06:13:09.782 INFO:teuthology.orchestra.run.smithi184.stdout:(4/9): jq-1.6-16.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-03-21T06:13:09.808 INFO:teuthology.orchestra.run.smithi184.stdout:(5/9): libcephsqlite-19.0.0-2165.g63e11116.el9. 405 kB/s | 158 kB 00:00 2024-03-21T06:13:10.008 INFO:teuthology.orchestra.run.smithi184.stdout:(6/9): libubsan-11.4.1-3.el9.x86_64.rpm 934 kB/s | 187 kB 00:00 2024-03-21T06:13:10.075 INFO:teuthology.orchestra.run.smithi184.stdout:(7/9): libasan-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 418 kB 00:00 2024-03-21T06:13:10.200 INFO:teuthology.orchestra.run.smithi184.stdout:(8/9): yaml-cpp-0.6.3-5.el9.x86_64.rpm 634 kB/s | 122 kB 00:00 2024-03-21T06:13:18.008 INFO:teuthology.orchestra.run.smithi184.stdout:(9/9): ceph-test-19.0.0-2165.g63e11116.el9.x86_ 14 MB/s | 123 MB 00:08 2024-03-21T06:13:18.010 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:13:18.011 INFO:teuthology.orchestra.run.smithi184.stdout:Total 14 MB/s | 124 MB 00:08 2024-03-21T06:13:18.050 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-21T06:13:18.071 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-21T06:13:18.071 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-21T06:13:18.908 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-21T06:13:18.908 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-21T06:13:20.069 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-21T06:13:20.131 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/9 2024-03-21T06:13:20.166 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libubsan-11.4.1-3.el9.x86_64 2/9 2024-03-21T06:13:20.199 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libasan-11.4.1-3.el9.x86_64 3/9 2024-03-21T06:13:20.243 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 4/9 2024-03-21T06:13:20.277 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 5/9 2024-03-21T06:13:20.315 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 6/9 2024-03-21T06:13:20.363 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : jq-1.6-16.el9.x86_64 7/9 2024-03-21T06:13:20.387 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephsqlite-2:19.0.0-2165.g63e11116.el9.x86_64 8/9 2024-03-21T06:13:37.207 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2165.g63e11116.el9.x86_64 8/9 2024-03-21T06:13:37.285 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-test-2:19.0.0-2165.g63e11116.el9.x86_64 9/9 2024-03-21T06:13:45.651 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-test-2:19.0.0-2165.g63e11116.el9.x86_64 9/9 2024-03-21T06:13:45.651 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-test-2:19.0.0-2165.g63e11116.el9.x86_64 1/9 2024-03-21T06:13:45.651 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephsqlite-2:19.0.0-2165.g63e11116.el9.x86_64 2/9 2024-03-21T06:13:45.652 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/9 2024-03-21T06:13:45.652 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/9 2024-03-21T06:13:45.652 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/9 2024-03-21T06:13:45.652 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : jq-1.6-16.el9.x86_64 6/9 2024-03-21T06:13:45.652 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libasan-11.4.1-3.el9.x86_64 7/9 2024-03-21T06:13:45.652 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libubsan-11.4.1-3.el9.x86_64 8/9 2024-03-21T06:13:46.057 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 9/9 2024-03-21T06:13:46.058 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:13:46.058 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-21T06:13:46.058 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:13:46.058 INFO:teuthology.orchestra.run.smithi184.stdout: jq-1.6-16.el9.x86_64 2024-03-21T06:13:46.058 INFO:teuthology.orchestra.run.smithi184.stdout: libasan-11.4.1-3.el9.x86_64 2024-03-21T06:13:46.058 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:13:46.058 INFO:teuthology.orchestra.run.smithi184.stdout: libubsan-11.4.1-3.el9.x86_64 2024-03-21T06:13:46.058 INFO:teuthology.orchestra.run.smithi184.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T06:13:46.058 INFO:teuthology.orchestra.run.smithi184.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T06:13:46.058 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T06:13:46.058 INFO:teuthology.orchestra.run.smithi184.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-03-21T06:13:46.059 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:13:46.059 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:13:46.356 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph 2024-03-21T06:13:46.912 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:13 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:13:47.061 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:13:47.064 INFO:teuthology.orchestra.run.smithi184.stdout:========================================================================================= 2024-03-21T06:13:47.064 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-03-21T06:13:47.064 INFO:teuthology.orchestra.run.smithi184.stdout:========================================================================================= 2024-03-21T06:13:47.064 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-21T06:13:47.064 INFO:teuthology.orchestra.run.smithi184.stdout: ceph x86_64 2:19.0.0-2165.g63e11116.el9 ceph 6.6 k 2024-03-21T06:13:47.064 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-03-21T06:13:47.064 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds x86_64 2:19.0.0-2165.g63e11116.el9 ceph 2.2 M 2024-03-21T06:13:47.065 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr x86_64 2:19.0.0-2165.g63e11116.el9 ceph 1.5 M 2024-03-21T06:13:47.065 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 248 k 2024-03-21T06:13:47.065 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon x86_64 2:19.0.0-2165.g63e11116.el9 ceph 4.9 M 2024-03-21T06:13:47.065 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd x86_64 2:19.0.0-2165.g63e11116.el9 ceph 119 M 2024-03-21T06:13:47.065 INFO:teuthology.orchestra.run.smithi184.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T06:13:47.065 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T06:13:47.065 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T06:13:47.065 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T06:13:47.065 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T06:13:47.065 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T06:13:47.066 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T06:13:47.066 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T06:13:47.066 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T06:13:47.066 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T06:13:47.066 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T06:13:47.066 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T06:13:47.066 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T06:13:47.066 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T06:13:47.066 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T06:13:47.066 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T06:13:47.066 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T06:13:47.066 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T06:13:47.066 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T06:13:47.067 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T06:13:47.067 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T06:13:47.067 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T06:13:47.067 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T06:13:47.067 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T06:13:47.067 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T06:13:47.067 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T06:13:47.067 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T06:13:47.067 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T06:13:47.067 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T06:13:47.068 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T06:13:47.068 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T06:13:47.068 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:13:47.068 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-21T06:13:47.068 INFO:teuthology.orchestra.run.smithi184.stdout:========================================================================================= 2024-03-21T06:13:47.068 INFO:teuthology.orchestra.run.smithi184.stdout:Install 37 Packages 2024-03-21T06:13:47.068 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:13:47.069 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 133 M 2024-03-21T06:13:47.069 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 1.4 G 2024-03-21T06:13:47.069 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-21T06:13:47.776 INFO:teuthology.orchestra.run.smithi184.stdout:(1/37): ceph-19.0.0-2165.g63e11116.el9.x86_64.r 31 kB/s | 6.6 kB 00:00 2024-03-21T06:13:48.001 INFO:teuthology.orchestra.run.smithi184.stdout:(2/37): ceph-mds-19.0.0-2165.g63e11116.el9.x86_ 5.1 MB/s | 2.2 MB 00:00 2024-03-21T06:13:48.060 INFO:teuthology.orchestra.run.smithi184.stdout:(3/37): ceph-mgr-19.0.0-2165.g63e11116.el9.x86_ 3.0 MB/s | 1.5 MB 00:00 2024-03-21T06:13:48.143 INFO:teuthology.orchestra.run.smithi184.stdout:(4/37): ceph-mgr-modules-core-19.0.0-2165.g63e1 2.9 MB/s | 248 kB 00:00 2024-03-21T06:13:48.302 INFO:teuthology.orchestra.run.smithi184.stdout:(5/37): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-21T06:13:48.386 INFO:teuthology.orchestra.run.smithi184.stdout:(6/37): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-03-21T06:13:48.436 INFO:teuthology.orchestra.run.smithi184.stdout:(7/37): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-21T06:13:48.486 INFO:teuthology.orchestra.run.smithi184.stdout:(8/37): python3-markupsafe-1.1.1-12.el9.x86_64. 634 kB/s | 32 kB 00:00 2024-03-21T06:13:48.528 INFO:teuthology.orchestra.run.smithi184.stdout:(9/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-21T06:13:48.562 INFO:teuthology.orchestra.run.smithi184.stdout:(10/37): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-03-21T06:13:48.612 INFO:teuthology.orchestra.run.smithi184.stdout:(11/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-21T06:13:48.679 INFO:teuthology.orchestra.run.smithi184.stdout:(12/37): python3-cryptography-36.0.1-4.el9.x86_ 17 MB/s | 1.1 MB 00:00 2024-03-21T06:13:48.704 INFO:teuthology.orchestra.run.smithi184.stdout:(13/37): python3-ply-3.11-14.el9.noarch.rpm 4.1 MB/s | 103 kB 00:00 2024-03-21T06:13:48.738 INFO:teuthology.orchestra.run.smithi184.stdout:(14/37): python3-pycparser-2.20-6.el9.noarch.rp 3.6 MB/s | 124 kB 00:00 2024-03-21T06:13:48.780 INFO:teuthology.orchestra.run.smithi184.stdout:(15/37): python3-urllib3-1.26.5-5.el9.noarch.rp 4.4 MB/s | 187 kB 00:00 2024-03-21T06:13:48.905 INFO:teuthology.orchestra.run.smithi184.stdout:(16/37): ceph-mon-19.0.0-2165.g63e11116.el9.x86 4.3 MB/s | 4.9 MB 00:01 2024-03-21T06:13:48.947 INFO:teuthology.orchestra.run.smithi184.stdout:(17/37): python3-requests-2.25.1-8.el9.noarch.r 748 kB/s | 125 kB 00:00 2024-03-21T06:13:49.089 INFO:teuthology.orchestra.run.smithi184.stdout:(18/37): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-03-21T06:13:49.117 INFO:teuthology.orchestra.run.smithi184.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 105 kB/s | 22 kB 00:00 2024-03-21T06:13:49.148 INFO:teuthology.orchestra.run.smithi184.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 741 kB/s | 43 kB 00:00 2024-03-21T06:13:49.232 INFO:teuthology.orchestra.run.smithi184.stdout:(21/37): python3-cherrypy-18.6.1-2.el9.noarch.r 4.2 MB/s | 358 kB 00:00 2024-03-21T06:13:49.265 INFO:teuthology.orchestra.run.smithi184.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 320 kB/s | 11 kB 00:00 2024-03-21T06:13:49.291 INFO:teuthology.orchestra.run.smithi184.stdout:(23/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 992 kB/s | 172 kB 00:00 2024-03-21T06:13:49.308 INFO:teuthology.orchestra.run.smithi184.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 421 kB/s | 18 kB 00:00 2024-03-21T06:13:49.325 INFO:teuthology.orchestra.run.smithi184.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 688 kB/s | 23 kB 00:00 2024-03-21T06:13:49.342 INFO:teuthology.orchestra.run.smithi184.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 578 kB/s | 19 kB 00:00 2024-03-21T06:13:49.367 INFO:teuthology.orchestra.run.smithi184.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 468 kB/s | 20 kB 00:00 2024-03-21T06:13:49.392 INFO:teuthology.orchestra.run.smithi184.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-03-21T06:13:49.417 INFO:teuthology.orchestra.run.smithi184.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-03-21T06:13:49.443 INFO:teuthology.orchestra.run.smithi184.stdout:(30/37): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-03-21T06:13:49.468 INFO:teuthology.orchestra.run.smithi184.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-03-21T06:13:49.493 INFO:teuthology.orchestra.run.smithi184.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-03-21T06:13:49.519 INFO:teuthology.orchestra.run.smithi184.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 711 kB/s | 36 kB 00:00 2024-03-21T06:13:49.544 INFO:teuthology.orchestra.run.smithi184.stdout:(34/37): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-03-21T06:13:49.586 INFO:teuthology.orchestra.run.smithi184.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 479 kB/s | 20 kB 00:00 2024-03-21T06:13:49.611 INFO:teuthology.orchestra.run.smithi184.stdout:(36/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-03-21T06:13:53.903 INFO:teuthology.orchestra.run.smithi184.stdout:(37/37): ceph-osd-19.0.0-2165.g63e11116.el9.x86 20 MB/s | 119 MB 00:05 2024-03-21T06:13:53.906 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:13:53.907 INFO:teuthology.orchestra.run.smithi184.stdout:Total 19 MB/s | 133 MB 00:06 2024-03-21T06:13:53.932 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-21T06:13:53.978 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-21T06:13:53.978 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-21T06:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-21T06:13:55.010 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-21T06:13:56.898 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-21T06:13:56.946 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-03-21T06:13:56.980 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-03-21T06:13:57.015 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-03-21T06:13:57.085 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-03-21T06:13:57.131 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/37 2024-03-21T06:13:57.163 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-03-21T06:13:57.247 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-03-21T06:13:57.325 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-03-21T06:13:57.366 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/37 2024-03-21T06:13:57.438 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-03-21T06:13:57.492 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/37 2024-03-21T06:13:57.530 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/37 2024-03-21T06:13:57.579 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/37 2024-03-21T06:13:57.668 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/37 2024-03-21T06:13:57.853 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/37 2024-03-21T06:13:57.902 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/37 2024-03-21T06:13:57.943 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/37 2024-03-21T06:13:57.983 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/37 2024-03-21T06:13:58.013 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/37 2024-03-21T06:13:58.088 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/37 2024-03-21T06:13:58.145 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/37 2024-03-21T06:13:58.156 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/37 2024-03-21T06:13:58.197 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/37 2024-03-21T06:13:58.223 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/37 2024-03-21T06:13:58.258 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/37 2024-03-21T06:13:58.347 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/37 2024-03-21T06:13:58.429 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/37 2024-03-21T06:13:58.462 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/37 2024-03-21T06:13:58.597 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/37 2024-03-21T06:13:58.716 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/37 2024-03-21T06:13:58.885 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2165.g63e11116.el9. 31/37 2024-03-21T06:13:58.910 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-2:19.0.0-2165.g63e11116.el9.x86_64 32/37 2024-03-21T06:13:58.936 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2165.g63e11116.el9.x86_64 32/37 2024-03-21T06:13:58.936 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T06:13:58.936 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T06:13:58.936 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T06:13:58.936 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T06:13:58.937 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:17.536 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 33/37 2024-03-21T06:14:17.568 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-osd-2:19.0.0-2165.g63e11116.el9.x86_64 34/37 2024-03-21T06:14:17.673 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-osd-2:19.0.0-2165.g63e11116.el9.x86_64 34/37 2024-03-21T06:14:17.674 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T06:14:17.674 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T06:14:17.674 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T06:14:17.674 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T06:14:17.674 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:18.069 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mon-2:19.0.0-2165.g63e11116.el9.x86_64 35/37 2024-03-21T06:14:18.094 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mon-2:19.0.0-2165.g63e11116.el9.x86_64 35/37 2024-03-21T06:14:18.094 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T06:14:18.094 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T06:14:18.094 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T06:14:18.094 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T06:14:18.094 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:18.328 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mds-2:19.0.0-2165.g63e11116.el9.x86_64 36/37 2024-03-21T06:14:18.352 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mds-2:19.0.0-2165.g63e11116.el9.x86_64 36/37 2024-03-21T06:14:18.353 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T06:14:18.353 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T06:14:18.353 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T06:14:18.353 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T06:14:18.353 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:18.417 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-2:19.0.0-2165.g63e11116.el9.x86_64 37/37 2024-03-21T06:14:28.498 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-2:19.0.0-2165.g63e11116.el9.x86_64 37/37 2024-03-21T06:14:28.498 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-2:19.0.0-2165.g63e11116.el9.x86_64 1/37 2024-03-21T06:14:28.498 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mds-2:19.0.0-2165.g63e11116.el9.x86_64 2/37 2024-03-21T06:14:28.498 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-2:19.0.0-2165.g63e11116.el9.x86_64 3/37 2024-03-21T06:14:28.498 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mon-2:19.0.0-2165.g63e11116.el9.x86_64 4/37 2024-03-21T06:14:28.499 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-osd-2:19.0.0-2165.g63e11116.el9.x86_64 5/37 2024-03-21T06:14:28.499 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2165.g63e11116.el9. 6/37 2024-03-21T06:14:28.499 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/37 2024-03-21T06:14:28.499 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 8/37 2024-03-21T06:14:28.499 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 9/37 2024-03-21T06:14:28.499 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 10/37 2024-03-21T06:14:28.499 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 11/37 2024-03-21T06:14:28.499 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 12/37 2024-03-21T06:14:28.499 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 13/37 2024-03-21T06:14:28.499 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 14/37 2024-03-21T06:14:28.499 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ply-3.11-14.el9.noarch 15/37 2024-03-21T06:14:28.499 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 16/37 2024-03-21T06:14:28.499 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 17/37 2024-03-21T06:14:28.500 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 18/37 2024-03-21T06:14:28.500 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-03-21T06:14:28.500 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 20/37 2024-03-21T06:14:28.500 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-03-21T06:14:28.500 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-03-21T06:14:28.500 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-03-21T06:14:28.500 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-03-21T06:14:28.500 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-03-21T06:14:28.500 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-03-21T06:14:28.500 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-03-21T06:14:28.501 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-03-21T06:14:28.501 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-03-21T06:14:28.501 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-03-21T06:14:28.501 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-03-21T06:14:28.502 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-03-21T06:14:28.502 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-03-21T06:14:28.502 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-03-21T06:14:28.502 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 35/37 2024-03-21T06:14:28.502 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-03-21T06:14:29.504 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-03-21T06:14:29.504 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:29.504 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-21T06:14:29.504 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:14:29.504 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:14:29.504 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:14:29.505 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:14:29.505 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:14:29.505 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:14:29.505 INFO:teuthology.orchestra.run.smithi184.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T06:14:29.505 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T06:14:29.505 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T06:14:29.505 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T06:14:29.505 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T06:14:29.505 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T06:14:29.505 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T06:14:29.505 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T06:14:29.505 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T06:14:29.505 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T06:14:29.506 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T06:14:29.506 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T06:14:29.506 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T06:14:29.506 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T06:14:29.506 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T06:14:29.506 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T06:14:29.506 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T06:14:29.506 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T06:14:29.506 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T06:14:29.506 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T06:14:29.506 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T06:14:29.506 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T06:14:29.506 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T06:14:29.507 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T06:14:29.507 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T06:14:29.507 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T06:14:29.507 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T06:14:29.507 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T06:14:29.507 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T06:14:29.507 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T06:14:29.507 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T06:14:29.507 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:29.507 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:14:29.787 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-base 2024-03-21T06:14:30.346 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:57 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:14:30.444 INFO:teuthology.orchestra.run.smithi184.stdout:Package ceph-base-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:14:30.497 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:14:30.498 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-03-21T06:14:30.498 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:14:30.568 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install cephadm 2024-03-21T06:14:31.142 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:58 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:14:31.289 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:14:31.290 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:14:31.290 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-03-21T06:14:31.290 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:14:31.291 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-21T06:14:31.291 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 748 k 2024-03-21T06:14:31.291 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:31.291 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-21T06:14:31.291 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:14:31.291 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-03-21T06:14:31.291 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:31.291 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 748 k 2024-03-21T06:14:31.291 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 755 k 2024-03-21T06:14:31.291 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-21T06:14:31.584 INFO:teuthology.orchestra.run.smithi184.stdout:cephadm-19.0.0-2165.g63e11116.el9.noarch.rpm 2.5 MB/s | 748 kB 00:00 2024-03-21T06:14:31.584 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:14:31.585 INFO:teuthology.orchestra.run.smithi184.stdout:Total 2.5 MB/s | 748 kB 00:00 2024-03-21T06:14:31.585 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-21T06:14:31.589 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-21T06:14:31.590 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-21T06:14:31.595 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-21T06:14:31.596 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-21T06:14:31.671 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-21T06:14:32.079 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:19.0.0-2165.g63e11116.el9.noarch 1/1 2024-03-21T06:14:32.164 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephadm-2:19.0.0-2165.g63e11116.el9.noarch 1/1 2024-03-21T06:14:32.563 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:19.0.0-2165.g63e11116.el9.noarch 1/1 2024-03-21T06:14:32.870 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephadm-2:19.0.0-2165.g63e11116.el9.noarch 1/1 2024-03-21T06:14:32.917 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:32.917 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-21T06:14:32.917 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:14:32.917 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:32.917 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:14:33.106 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-immutable-object-cache 2024-03-21T06:14:33.673 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:00 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:14:33.827 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:14:33.828 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:14:33.828 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-03-21T06:14:33.829 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:14:33.829 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-21T06:14:33.829 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2165.g63e11116.el9 ceph 147 k 2024-03-21T06:14:33.829 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:33.829 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-21T06:14:33.829 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:14:33.829 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-03-21T06:14:33.829 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:33.829 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 147 k 2024-03-21T06:14:33.829 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 481 k 2024-03-21T06:14:33.829 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-21T06:14:34.056 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-immutable-object-cache-19.0.0-2165.g63e111 649 kB/s | 147 kB 00:00 2024-03-21T06:14:34.057 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:14:34.057 INFO:teuthology.orchestra.run.smithi184.stdout:Total 644 kB/s | 147 kB 00:00 2024-03-21T06:14:34.057 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-21T06:14:34.065 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-21T06:14:34.065 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-21T06:14:34.117 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-21T06:14:34.117 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-21T06:14:34.252 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-21T06:14:34.324 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2165.g63e11116. 1/1 2024-03-21T06:14:34.348 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2165.g63e11116. 1/1 2024-03-21T06:14:34.348 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T06:14:34.348 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T06:14:34.349 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:35.286 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2165.g63e11116. 1/1 2024-03-21T06:14:35.287 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:35.287 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-21T06:14:35.287 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:14:35.287 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:35.287 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:14:35.540 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr 2024-03-21T06:14:36.117 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:03 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:14:36.215 INFO:teuthology.orchestra.run.smithi184.stdout:Package ceph-mgr-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:14:36.267 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:14:36.268 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-03-21T06:14:36.268 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:14:36.334 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-dashboard 2024-03-21T06:14:36.890 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:03 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:14:37.038 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:14:37.039 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-03-21T06:14:37.039 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-03-21T06:14:37.039 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-03-21T06:14:37.039 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-21T06:14:37.039 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 1.8 M 2024-03-21T06:14:37.039 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-03-21T06:14:37.039 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T06:14:37.039 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 34 k 2024-03-21T06:14:37.039 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 16 k 2024-03-21T06:14:37.039 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T06:14:37.039 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T06:14:37.039 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T06:14:37.039 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T06:14:37.040 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T06:14:37.040 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T06:14:37.040 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T06:14:37.040 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:37.040 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-21T06:14:37.040 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================== 2024-03-21T06:14:37.040 INFO:teuthology.orchestra.run.smithi184.stdout:Install 11 Packages 2024-03-21T06:14:37.040 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:37.041 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 5.9 M 2024-03-21T06:14:37.041 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 23 M 2024-03-21T06:14:37.041 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-21T06:14:37.593 INFO:teuthology.orchestra.run.smithi184.stdout:(1/11): ceph-grafana-dashboards-19.0.0-2165.g63 176 kB/s | 34 kB 00:00 2024-03-21T06:14:37.652 INFO:teuthology.orchestra.run.smithi184.stdout:(2/11): ceph-prometheus-alerts-19.0.0-2165.g63e 65 kB/s | 16 kB 00:00 2024-03-21T06:14:37.722 INFO:teuthology.orchestra.run.smithi184.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.9 MB/s | 238 kB 00:00 2024-03-21T06:14:37.927 INFO:teuthology.orchestra.run.smithi184.stdout:(4/11): protobuf-compiler-3.14.0-13.el9.x86_64. 3.1 MB/s | 863 kB 00:00 2024-03-21T06:14:38.044 INFO:teuthology.orchestra.run.smithi184.stdout:(5/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.8 MB/s | 548 kB 00:00 2024-03-21T06:14:38.070 INFO:teuthology.orchestra.run.smithi184.stdout:(6/11): grpc-data-1.46.7-10.el9.noarch.rpm 137 kB/s | 19 kB 00:00 2024-03-21T06:14:38.186 INFO:teuthology.orchestra.run.smithi184.stdout:(7/11): python3-grpcio-1.46.7-10.el9.x86_64.rpm 14 MB/s | 2.0 MB 00:00 2024-03-21T06:14:38.212 INFO:teuthology.orchestra.run.smithi184.stdout:(8/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.0 MB/s | 144 kB 00:00 2024-03-21T06:14:38.237 INFO:teuthology.orchestra.run.smithi184.stdout:(9/11): python3-repoze-lru-0.7-16.el9.noarch.rp 612 kB/s | 31 kB 00:00 2024-03-21T06:14:38.296 INFO:teuthology.orchestra.run.smithi184.stdout:(10/11): python3-routes-2.5.1-5.el9.noarch.rpm 2.2 MB/s | 188 kB 00:00 2024-03-21T06:14:38.871 INFO:teuthology.orchestra.run.smithi184.stdout:(11/11): ceph-mgr-dashboard-19.0.0-2165.g63e111 1.2 MB/s | 1.8 MB 00:01 2024-03-21T06:14:38.875 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:14:38.875 INFO:teuthology.orchestra.run.smithi184.stdout:Total 3.2 MB/s | 5.9 MB 00:01 2024-03-21T06:14:38.894 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-21T06:14:38.911 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-21T06:14:38.911 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-21T06:14:39.084 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-21T06:14:39.084 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-21T06:14:39.563 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-21T06:14:39.635 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-03-21T06:14:39.682 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-03-21T06:14:39.793 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-03-21T06:14:39.924 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-03-21T06:14:39.977 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-03-21T06:14:40.060 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-03-21T06:14:40.112 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-03-21T06:14:40.141 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-03-21T06:14:40.173 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2165.g63e11116.el9 9/11 2024-03-21T06:14:40.417 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2165.g63e11116.el 10/11 2024-03-21T06:14:40.464 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2165.g63e11116.el9.noa 11/11 2024-03-21T06:14:41.222 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2165.g63e11116.el9.noa 11/11 2024-03-21T06:14:41.222 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2165.g63e11116.el 1/11 2024-03-21T06:14:41.223 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2165.g63e11116.el9.noa 2/11 2024-03-21T06:14:41.223 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2165.g63e11116.el9 3/11 2024-03-21T06:14:41.223 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-03-21T06:14:41.223 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-03-21T06:14:41.223 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-03-21T06:14:41.223 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-03-21T06:14:41.223 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-03-21T06:14:41.223 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-03-21T06:14:41.223 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-03-21T06:14:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-03-21T06:14:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:41.558 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-21T06:14:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T06:14:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:14:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:14:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:14:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T06:14:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T06:14:41.558 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T06:14:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T06:14:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T06:14:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T06:14:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T06:14:41.559 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:41.559 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:14:41.818 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-21T06:14:42.379 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:09 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:14:42.530 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:14:42.531 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================ 2024-03-21T06:14:42.531 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-03-21T06:14:42.532 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================ 2024-03-21T06:14:42.532 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-21T06:14:42.532 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 7.4 M 2024-03-21T06:14:42.532 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-03-21T06:14:42.532 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T06:14:42.532 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T06:14:42.532 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T06:14:42.532 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T06:14:42.532 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T06:14:42.532 INFO:teuthology.orchestra.run.smithi184.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T06:14:42.532 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T06:14:42.532 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T06:14:42.533 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T06:14:42.533 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T06:14:42.533 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T06:14:42.533 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:42.533 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-21T06:14:42.533 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================ 2024-03-21T06:14:42.533 INFO:teuthology.orchestra.run.smithi184.stdout:Install 12 Packages 2024-03-21T06:14:42.533 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:42.534 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 37 M 2024-03-21T06:14:42.534 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 209 M 2024-03-21T06:14:42.534 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-21T06:14:42.810 INFO:teuthology.orchestra.run.smithi184.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 204 kB/s | 32 kB 00:00 2024-03-21T06:14:42.835 INFO:teuthology.orchestra.run.smithi184.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 692 kB/s | 17 kB 00:00 2024-03-21T06:14:42.860 INFO:teuthology.orchestra.run.smithi184.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.3 MB/s | 34 kB 00:00 2024-03-21T06:14:43.019 INFO:teuthology.orchestra.run.smithi184.stdout:(4/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 30 MB/s | 4.7 MB 00:00 2024-03-21T06:14:43.145 INFO:teuthology.orchestra.run.smithi184.stdout:(5/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 39 MB/s | 4.8 MB 00:00 2024-03-21T06:14:43.203 INFO:teuthology.orchestra.run.smithi184.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.5 MB/s | 3.0 MB 00:00 2024-03-21T06:14:43.228 INFO:teuthology.orchestra.run.smithi184.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.6 MB/s | 217 kB 00:00 2024-03-21T06:14:43.445 INFO:teuthology.orchestra.run.smithi184.stdout:(8/12): ceph-mgr-diskprediction-local-19.0.0-21 9.4 MB/s | 7.4 MB 00:00 2024-03-21T06:14:43.621 INFO:teuthology.orchestra.run.smithi184.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-03-21T06:14:44.246 INFO:teuthology.orchestra.run.smithi184.stdout:(10/12): python3-devel-3.9.17-2.el9.x86_64.rpm 409 kB/s | 245 kB 00:00 2024-03-21T06:14:44.429 INFO:teuthology.orchestra.run.smithi184.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 199 kB/s | 195 kB 00:00 2024-03-21T06:14:45.029 INFO:teuthology.orchestra.run.smithi184.stdout:(12/12): libgfortran-11.4.1-3.el9.x86_64.rpm 446 kB/s | 803 kB 00:01 2024-03-21T06:14:45.033 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:14:45.033 INFO:teuthology.orchestra.run.smithi184.stdout:Total 15 MB/s | 37 MB 00:02 2024-03-21T06:14:45.078 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-21T06:14:45.106 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-21T06:14:45.106 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-21T06:14:45.469 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-21T06:14:45.469 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-21T06:14:46.178 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-21T06:14:46.261 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-21T06:14:46.300 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-21T06:14:46.393 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-21T06:14:46.428 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-21T06:14:46.614 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-21T06:14:46.644 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-21T06:14:46.721 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-21T06:14:47.366 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-21T06:14:47.473 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-21T06:14:48.712 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-21T06:14:49.759 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-21T06:14:49.785 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2165.g63e11 12/12 2024-03-21T06:14:52.319 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2165.g63e11 12/12 2024-03-21T06:14:52.319 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2165.g63e11 1/12 2024-03-21T06:14:52.319 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-21T06:14:52.319 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-21T06:14:52.319 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-21T06:14:52.319 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-21T06:14:52.319 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-21T06:14:52.319 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-21T06:14:52.320 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-21T06:14:52.320 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-21T06:14:52.320 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-21T06:14:52.320 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-21T06:14:52.803 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-21T06:14:52.803 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:52.804 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-21T06:14:52.804 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:14:52.804 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T06:14:52.804 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T06:14:52.804 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T06:14:52.804 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T06:14:52.804 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T06:14:52.804 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T06:14:52.804 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T06:14:52.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T06:14:52.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T06:14:52.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T06:14:52.804 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T06:14:52.804 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:52.805 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:14:53.225 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-rook 2024-03-21T06:14:53.805 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:20 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:14:53.957 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:14:53.958 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-03-21T06:14:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-03-21T06:14:53.958 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-03-21T06:14:53.958 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-21T06:14:53.958 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 49 k 2024-03-21T06:14:53.959 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-03-21T06:14:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T06:14:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T06:14:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T06:14:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T06:14:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T06:14:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T06:14:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T06:14:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T06:14:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T06:14:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T06:14:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T06:14:53.959 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T06:14:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:53.960 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-21T06:14:53.960 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-03-21T06:14:53.960 INFO:teuthology.orchestra.run.smithi184.stdout:Install 13 Packages 2024-03-21T06:14:53.960 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:53.960 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 2.0 M 2024-03-21T06:14:53.961 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 26 M 2024-03-21T06:14:53.961 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-21T06:14:54.318 INFO:teuthology.orchestra.run.smithi184.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 134 kB/s | 18 kB 00:00 2024-03-21T06:14:54.343 INFO:teuthology.orchestra.run.smithi184.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 152 kB/s | 24 kB 00:00 2024-03-21T06:14:54.385 INFO:teuthology.orchestra.run.smithi184.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 174 kB 00:00 2024-03-21T06:14:54.411 INFO:teuthology.orchestra.run.smithi184.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-21T06:14:54.436 INFO:teuthology.orchestra.run.smithi184.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-03-21T06:14:54.461 INFO:teuthology.orchestra.run.smithi184.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 858 kB/s | 43 kB 00:00 2024-03-21T06:14:54.487 INFO:teuthology.orchestra.run.smithi184.stdout:(7/13): ceph-mgr-rook-19.0.0-2165.g63e11116.el9 163 kB/s | 49 kB 00:00 2024-03-21T06:14:54.699 INFO:teuthology.orchestra.run.smithi184.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 124 kB/s | 32 kB 00:00 2024-03-21T06:14:54.720 INFO:teuthology.orchestra.run.smithi184.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 58 kB/s | 15 kB 00:00 2024-03-21T06:14:54.887 INFO:teuthology.orchestra.run.smithi184.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 355 kB/s | 59 kB 00:00 2024-03-21T06:14:54.913 INFO:teuthology.orchestra.run.smithi184.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 476 kB/s | 203 kB 00:00 2024-03-21T06:14:55.063 INFO:teuthology.orchestra.run.smithi184.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 511 kB/s | 90 kB 00:00 2024-03-21T06:14:55.096 INFO:teuthology.orchestra.run.smithi184.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.5 MB/s | 1.0 MB 00:00 2024-03-21T06:14:55.098 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:14:55.098 INFO:teuthology.orchestra.run.smithi184.stdout:Total 1.8 MB/s | 2.0 MB 00:01 2024-03-21T06:14:55.099 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-21T06:14:55.113 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-21T06:14:55.113 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-21T06:14:55.204 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-21T06:14:55.206 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-21T06:14:55.719 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-21T06:14:55.883 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-21T06:14:55.947 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-21T06:14:56.002 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-21T06:14:56.040 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-21T06:14:56.081 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-21T06:14:56.175 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-21T06:14:56.283 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-21T06:14:56.333 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-21T06:14:57.053 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-21T06:14:57.096 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-21T06:14:57.130 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-21T06:14:57.178 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-21T06:14:57.200 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-rook-2:19.0.0-2165.g63e11116.el9.noarch 13/13 2024-03-21T06:14:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2165.g63e11116.el9.noarch 13/13 2024-03-21T06:14:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2165.g63e11116.el9.noarch 1/13 2024-03-21T06:14:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-21T06:14:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-21T06:14:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-21T06:14:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-21T06:14:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-21T06:14:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-21T06:14:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-21T06:14:58.003 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-21T06:14:58.004 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-21T06:14:58.004 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-21T06:14:58.004 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-21T06:14:58.282 INFO:teuthology.orchestra.run.smithi096.stdout:[MIRROR] gperftools-libs-2.9.1-2.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://epel.stl.us.ssimn.org/9/Everything/x86_64/Packages/g/gperftools-libs-2.9.1-2.el9.x86_64.rpm [Failed to connect to epel.stl.us.ssimn.org port 80: Connection timed out] 2024-03-21T06:14:58.374 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-21T06:14:58.374 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:58.375 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-21T06:14:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:14:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T06:14:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T06:14:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T06:14:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T06:14:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T06:14:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T06:14:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T06:14:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T06:14:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T06:14:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T06:14:58.375 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T06:14:58.376 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T06:14:58.376 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:58.376 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:14:58.631 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-mgr-cephadm 2024-03-21T06:14:59.192 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:26 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:14:59.344 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:14:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-03-21T06:14:59.345 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-03-21T06:14:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-03-21T06:14:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-21T06:14:59.345 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 139 k 2024-03-21T06:14:59.345 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-03-21T06:14:59.345 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T06:14:59.345 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T06:14:59.345 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T06:14:59.345 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T06:14:59.346 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T06:14:59.346 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:59.346 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-21T06:14:59.346 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================== 2024-03-21T06:14:59.346 INFO:teuthology.orchestra.run.smithi184.stdout:Install 6 Packages 2024-03-21T06:14:59.346 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:14:59.346 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 6.8 M 2024-03-21T06:14:59.347 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 33 M 2024-03-21T06:14:59.347 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-21T06:15:00.089 INFO:teuthology.orchestra.run.smithi184.stdout:(1/6): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2024-03-21T06:15:00.114 INFO:teuthology.orchestra.run.smithi184.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 869 kB/s | 248 kB 00:00 2024-03-21T06:15:00.242 INFO:teuthology.orchestra.run.smithi184.stdout:(3/6): ceph-mgr-cephadm-19.0.0-2165.g63e11116.e 336 kB/s | 139 kB 00:00 2024-03-21T06:15:00.450 INFO:teuthology.orchestra.run.smithi184.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 172 kB/s | 58 kB 00:00 2024-03-21T06:15:00.525 INFO:teuthology.orchestra.run.smithi184.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 180 kB/s | 51 kB 00:00 2024-03-21T06:15:00.576 INFO:teuthology.orchestra.run.smithi184.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.1 MB/s | 548 kB 00:00 2024-03-21T06:15:00.579 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:15:00.579 INFO:teuthology.orchestra.run.smithi184.stdout:Total 5.5 MB/s | 6.8 MB 00:01 2024-03-21T06:15:00.591 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-21T06:15:00.601 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-21T06:15:00.601 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-21T06:15:00.690 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-21T06:15:00.691 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-21T06:15:00.983 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-21T06:15:01.156 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-21T06:15:01.210 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-21T06:15:01.721 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-21T06:15:01.842 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-21T06:15:01.909 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-21T06:15:01.939 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2165.g63e11116.el9.noarch 6/6 2024-03-21T06:15:02.696 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2165.g63e11116.el9.noarch 6/6 2024-03-21T06:15:02.697 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2165.g63e11116.el9.noarch 1/6 2024-03-21T06:15:02.697 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-21T06:15:02.697 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-21T06:15:02.697 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-21T06:15:02.697 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-21T06:15:03.007 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-21T06:15:03.007 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:03.008 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-21T06:15:03.008 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:15:03.008 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T06:15:03.008 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T06:15:03.008 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T06:15:03.008 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T06:15:03.008 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T06:15:03.008 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:03.008 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:15:03.264 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-fuse 2024-03-21T06:15:03.828 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:30 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:15:03.978 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:15:03.978 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:03.978 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-03-21T06:15:03.978 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:03.979 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-21T06:15:03.979 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse x86_64 2:19.0.0-2165.g63e11116.el9 ceph 861 k 2024-03-21T06:15:03.979 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-03-21T06:15:03.979 INFO:teuthology.orchestra.run.smithi184.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T06:15:03.979 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:03.979 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-21T06:15:03.979 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:03.979 INFO:teuthology.orchestra.run.smithi184.stdout:Install 2 Packages 2024-03-21T06:15:03.979 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:03.980 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 940 k 2024-03-21T06:15:03.980 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 3.1 M 2024-03-21T06:15:03.980 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-21T06:15:04.103 INFO:teuthology.orchestra.run.smithi184.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 641 kB/s | 79 kB 00:00 2024-03-21T06:15:04.295 INFO:teuthology.orchestra.run.smithi184.stdout:(2/2): ceph-fuse-19.0.0-2165.g63e11116.el9.x86_ 2.7 MB/s | 861 kB 00:00 2024-03-21T06:15:04.298 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:15:04.298 INFO:teuthology.orchestra.run.smithi184.stdout:Total 2.9 MB/s | 940 kB 00:00 2024-03-21T06:15:04.298 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-21T06:15:04.316 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-21T06:15:04.316 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-21T06:15:04.388 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-21T06:15:04.389 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-21T06:15:04.571 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-21T06:15:04.714 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-21T06:15:04.777 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-fuse-2:19.0.0-2165.g63e11116.el9.x86_64 2/2 2024-03-21T06:15:05.486 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2165.g63e11116.el9.x86_64 2/2 2024-03-21T06:15:05.486 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-fuse-2:19.0.0-2165.g63e11116.el9.x86_64 1/2 2024-03-21T06:15:05.846 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-21T06:15:05.846 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:05.846 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-21T06:15:05.846 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse-2:19.0.0-2165.g63e11116.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-21T06:15:05.846 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:05.846 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:15:06.079 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-volume 2024-03-21T06:15:06.660 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:33 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:15:06.817 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:15:06.818 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:06.818 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-03-21T06:15:06.818 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:06.818 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-21T06:15:06.818 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 264 k 2024-03-21T06:15:06.818 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-03-21T06:15:06.819 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T06:15:06.819 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:06.819 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-21T06:15:06.819 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:06.819 INFO:teuthology.orchestra.run.smithi184.stdout:Install 2 Packages 2024-03-21T06:15:06.819 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:06.819 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 332 k 2024-03-21T06:15:06.820 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 1.5 M 2024-03-21T06:15:06.820 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-21T06:15:06.941 INFO:teuthology.orchestra.run.smithi184.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 567 kB/s | 69 kB 00:00 2024-03-21T06:15:07.091 INFO:teuthology.orchestra.run.smithi184.stdout:(2/2): ceph-volume-19.0.0-2165.g63e11116.el9.no 972 kB/s | 264 kB 00:00 2024-03-21T06:15:07.092 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:15:07.092 INFO:teuthology.orchestra.run.smithi184.stdout:Total 1.2 MB/s | 332 kB 00:00 2024-03-21T06:15:07.092 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-21T06:15:07.101 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-21T06:15:07.101 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-21T06:15:07.139 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-21T06:15:07.139 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-21T06:15:07.356 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-21T06:15:07.525 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-03-21T06:15:07.569 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-volume-2:19.0.0-2165.g63e11116.el9.noarch 2/2 2024-03-21T06:15:07.584 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-volume-2:19.0.0-2165.g63e11116.el9.noarch 2/2 2024-03-21T06:15:07.584 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T06:15:07.585 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T06:15:07.585 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:08.276 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-volume-2:19.0.0-2165.g63e11116.el9.noarch 1/2 2024-03-21T06:15:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-03-21T06:15:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:08.608 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-21T06:15:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:15:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T06:15:08.608 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:08.608 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:15:08.922 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install librados-devel 2024-03-21T06:15:09.499 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:36 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:15:09.648 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:15:09.649 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:09.649 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-03-21T06:15:09.650 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:09.650 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-21T06:15:09.650 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel x86_64 2:19.0.0-2165.g63e11116.el9 ceph 127 k 2024-03-21T06:15:09.650 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:09.650 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-21T06:15:09.650 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:09.650 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-03-21T06:15:09.650 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:09.650 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 127 k 2024-03-21T06:15:09.650 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 457 k 2024-03-21T06:15:09.650 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-21T06:15:09.881 INFO:teuthology.orchestra.run.smithi184.stdout:librados-devel-19.0.0-2165.g63e11116.el9.x86_64 550 kB/s | 127 kB 00:00 2024-03-21T06:15:09.882 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:15:09.882 INFO:teuthology.orchestra.run.smithi184.stdout:Total 546 kB/s | 127 kB 00:00 2024-03-21T06:15:09.882 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-21T06:15:09.887 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-21T06:15:09.887 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-21T06:15:09.943 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-21T06:15:09.943 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-21T06:15:10.083 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-21T06:15:10.188 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librados-devel-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:15:10.628 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados-devel-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:15:10.931 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados-devel-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:15:10.931 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:10.931 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-21T06:15:10.931 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:15:10.931 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:10.931 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:15:11.191 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install libcephfs2 2024-03-21T06:15:11.804 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:38 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:15:11.905 INFO:teuthology.orchestra.run.smithi184.stdout:Package libcephfs2-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:15:11.957 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:15:11.958 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-03-21T06:15:11.958 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:15:12.014 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install libcephfs-devel 2024-03-21T06:15:12.605 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:39 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:15:12.754 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:15:12.754 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:12.755 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-03-21T06:15:12.755 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:12.755 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-21T06:15:12.755 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel x86_64 2:19.0.0-2165.g63e11116.el9 ceph 31 k 2024-03-21T06:15:12.755 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:12.755 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-21T06:15:12.755 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:12.755 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-03-21T06:15:12.755 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:12.755 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 31 k 2024-03-21T06:15:12.755 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 140 k 2024-03-21T06:15:12.755 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-21T06:15:12.935 INFO:teuthology.orchestra.run.smithi184.stdout:libcephfs-devel-19.0.0-2165.g63e11116.el9.x86_6 175 kB/s | 31 kB 00:00 2024-03-21T06:15:12.935 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:15:12.936 INFO:teuthology.orchestra.run.smithi184.stdout:Total 174 kB/s | 31 kB 00:00 2024-03-21T06:15:12.936 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-21T06:15:12.939 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-21T06:15:12.939 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-21T06:15:12.975 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-21T06:15:12.975 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-21T06:15:13.066 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-21T06:15:13.183 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs-devel-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:15:13.543 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:15:13.801 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs-devel-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:15:13.801 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:13.801 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-21T06:15:13.801 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:15:13.801 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:13.801 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:15:13.956 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install librados2 2024-03-21T06:15:14.519 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:41 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:15:14.615 INFO:teuthology.orchestra.run.smithi184.stdout:Package librados2-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:15:14.667 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:15:14.668 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-03-21T06:15:14.668 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:15:14.731 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install librbd1 2024-03-21T06:15:15.291 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:42 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:15:15.387 INFO:teuthology.orchestra.run.smithi184.stdout:Package librbd1-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:15:15.438 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:15:15.439 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-03-21T06:15:15.439 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:15:15.499 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-rados 2024-03-21T06:15:16.070 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:43 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:15:16.170 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-rados-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:15:16.222 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:15:16.223 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-03-21T06:15:16.223 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:15:16.281 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-rgw 2024-03-21T06:15:16.852 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:43 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:15:16.950 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-rgw-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:15:17.002 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:15:17.003 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-03-21T06:15:17.003 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:15:17.068 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-cephfs 2024-03-21T06:15:17.629 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:44 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:15:17.726 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-cephfs-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:15:17.778 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:15:17.779 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-03-21T06:15:17.779 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:15:17.842 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install python3-rbd 2024-03-21T06:15:18.411 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:45 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:15:18.508 INFO:teuthology.orchestra.run.smithi184.stdout:Package python3-rbd-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:15:18.560 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:15:18.561 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2024-03-21T06:15:18.561 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:15:18.623 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install rbd-fuse 2024-03-21T06:15:19.183 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:46 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:15:19.332 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:15:19.333 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:19.333 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-03-21T06:15:19.333 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:19.333 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-21T06:15:19.333 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse x86_64 2:19.0.0-2165.g63e11116.el9 ceph 86 k 2024-03-21T06:15:19.333 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:19.333 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-21T06:15:19.334 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:19.334 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-03-21T06:15:19.334 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:19.334 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 86 k 2024-03-21T06:15:19.334 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 256 k 2024-03-21T06:15:19.334 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-21T06:15:19.545 INFO:teuthology.orchestra.run.smithi184.stdout:rbd-fuse-19.0.0-2165.g63e11116.el9.x86_64.rpm 408 kB/s | 86 kB 00:00 2024-03-21T06:15:19.546 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:15:19.546 INFO:teuthology.orchestra.run.smithi184.stdout:Total 406 kB/s | 86 kB 00:00 2024-03-21T06:15:19.546 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-21T06:15:19.553 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-21T06:15:19.553 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-21T06:15:19.608 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-21T06:15:19.608 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-21T06:15:19.749 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-21T06:15:19.843 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-fuse-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:15:20.276 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:15:20.575 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-fuse-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:15:20.576 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:20.576 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-21T06:15:20.576 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:15:20.576 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:20.576 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:15:20.852 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install rbd-mirror 2024-03-21T06:15:21.440 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:48 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:15:21.595 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:15:21.595 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:21.595 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repo Size 2024-03-21T06:15:21.595 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:21.596 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-21T06:15:21.596 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror x86_64 2:19.0.0-2165.g63e11116.el9 ceph 3.0 M 2024-03-21T06:15:21.596 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:21.596 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-21T06:15:21.596 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:21.596 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-03-21T06:15:21.596 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:21.596 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 3.0 M 2024-03-21T06:15:21.597 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 14 M 2024-03-21T06:15:21.597 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-21T06:15:22.592 INFO:teuthology.orchestra.run.smithi184.stdout:rbd-mirror-19.0.0-2165.g63e11116.el9.x86_64.rpm 3.1 MB/s | 3.0 MB 00:00 2024-03-21T06:15:22.618 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:15:22.618 INFO:teuthology.orchestra.run.smithi184.stdout:Total 3.1 MB/s | 3.0 MB 00:00 2024-03-21T06:15:22.618 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-21T06:15:22.618 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-21T06:15:22.619 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-21T06:15:22.676 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-21T06:15:22.676 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-21T06:15:23.110 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-21T06:15:23.192 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-mirror-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:15:23.218 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:15:23.218 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T06:15:23.218 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T06:15:23.218 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T06:15:23.218 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T06:15:23.218 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:24.401 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-mirror-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:15:24.401 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:24.401 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-21T06:15:24.401 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:15:24.401 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:24.401 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:15:24.655 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install rbd-nbd 2024-03-21T06:15:25.228 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:52 ago on Thu 21 Mar 2024 06:12:33 AM UTC. 2024-03-21T06:15:25.376 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-03-21T06:15:25.376 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:25.377 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2024-03-21T06:15:25.377 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:25.377 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-03-21T06:15:25.377 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd x86_64 2:19.0.0-2165.g63e11116.el9 ceph 169 k 2024-03-21T06:15:25.377 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:25.377 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-03-21T06:15:25.377 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2024-03-21T06:15:25.377 INFO:teuthology.orchestra.run.smithi184.stdout:Install 1 Package 2024-03-21T06:15:25.377 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:25.377 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 169 k 2024-03-21T06:15:25.377 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 541 k 2024-03-21T06:15:25.377 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-03-21T06:15:25.746 INFO:teuthology.orchestra.run.smithi184.stdout:rbd-nbd-19.0.0-2165.g63e11116.el9.x86_64.rpm 460 kB/s | 169 kB 00:00 2024-03-21T06:15:25.746 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:15:25.747 INFO:teuthology.orchestra.run.smithi184.stdout:Total 458 kB/s | 169 kB 00:00 2024-03-21T06:15:25.747 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-03-21T06:15:25.754 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-03-21T06:15:25.754 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-03-21T06:15:25.809 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-03-21T06:15:25.810 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-03-21T06:15:25.959 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-03-21T06:15:26.061 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-nbd-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:15:26.492 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:15:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-nbd-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:15:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:26.791 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-03-21T06:15:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:15:26.791 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:15:26.791 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-03-21T06:15:27.030 DEBUG:teuthology.parallel:result is None 2024-03-21T06:17:09.249 INFO:teuthology.orchestra.run.smithi096.stdout:[MIRROR] gperftools-libs-2.9.1-2.el9.x86_64.rpm: Curl error (28): Timeout was reached for https://epel.stl.us.ssimn.org/9/Everything/x86_64/Packages/g/gperftools-libs-2.9.1-2.el9.x86_64.rpm [Failed to connect to epel.stl.us.ssimn.org port 443: Connection timed out] 2024-03-21T06:17:09.607 INFO:teuthology.orchestra.run.smithi096.stdout:(32/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.2 kB/s | 309 kB 04:22 2024-03-21T06:17:09.607 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:17:09.607 INFO:teuthology.orchestra.run.smithi096.stdout:Total 212 kB/s | 56 MB 04:31 2024-03-21T06:17:09.638 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-21T06:17:09.676 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-21T06:17:09.676 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-21T06:17:10.144 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-21T06:17:10.144 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-21T06:17:11.605 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-21T06:17:11.971 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-03-21T06:17:12.059 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librados2-2:19.0.0-2165.g63e11116.el9.x86_64 2/34 2024-03-21T06:17:12.149 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:19.0.0-2165.g63e11116.el9.x86_64 2/34 2024-03-21T06:17:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rados-2:19.0.0-2165.g63e11116.el9.x86_64 3/34 2024-03-21T06:17:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs2-2:19.0.0-2165.g63e11116.el9.x86_64 4/34 2024-03-21T06:17:12.342 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs2-2:19.0.0-2165.g63e11116.el9.x86_64 4/34 2024-03-21T06:17:12.372 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/34 2024-03-21T06:17:12.397 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/34 2024-03-21T06:17:12.420 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 7/34 2024-03-21T06:17:12.435 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 7/34 2024-03-21T06:17:12.647 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 8/34 2024-03-21T06:17:12.676 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 8/34 2024-03-21T06:17:12.705 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/34 2024-03-21T06:17:12.750 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-argparse-2:19.0.0-2165.g63e11116.el9. 10/34 2024-03-21T06:17:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cephfs-2:19.0.0-2165.g63e11116.el9.x86_64 11/34 2024-03-21T06:17:12.846 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 12/34 2024-03-21T06:17:12.875 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libradosstriper1-2:19.0.0-2165.g63e11116.el9.x86_6 13/34 2024-03-21T06:17:12.957 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2165.g63e11116.el9.x86_6 13/34 2024-03-21T06:17:12.992 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : thrift-0.15.0-2.el9.x86_64 14/34 2024-03-21T06:17:13.100 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-03-21T06:17:13.133 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-03-21T06:17:13.174 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-03-21T06:17:13.206 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-03-21T06:17:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-03-21T06:17:13.268 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-03-21T06:17:13.602 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-03-21T06:17:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librbd1-2:19.0.0-2165.g63e11116.el9.x86_64 22/34 2024-03-21T06:17:13.722 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:19.0.0-2165.g63e11116.el9.x86_64 22/34 2024-03-21T06:17:13.753 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rbd-2:19.0.0-2165.g63e11116.el9.x86_64 23/34 2024-03-21T06:17:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-03-21T06:17:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-03-21T06:17:14.284 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librgw2-2:19.0.0-2165.g63e11116.el9.x86_64 26/34 2024-03-21T06:17:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librgw2-2:19.0.0-2165.g63e11116.el9.x86_64 26/34 2024-03-21T06:17:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rgw-2:19.0.0-2165.g63e11116.el9.x86_64 27/34 2024-03-21T06:17:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-common-2:19.0.0-2165.g63e11116.el9.x8 28/34 2024-03-21T06:17:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.0.0-2165.g63e11116.el9.x86_64 29/34 2024-03-21T06:17:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-common-2:19.0.0-2165.g63e11116.el9.x86_64 29/34 2024-03-21T06:17:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.0.0-2165.g63e11116.el9.x86_64 29/34 2024-03-21T06:17:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-base-2:19.0.0-2165.g63e11116.el9.x86_64 30/34 2024-03-21T06:17:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-base-2:19.0.0-2165.g63e11116.el9.x86_64 30/34 2024-03-21T06:17:17.383 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T06:17:17.383 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T06:17:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:17:17.420 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-selinux-2:19.0.0-2165.g63e11116.el9.x86_64 31/34 2024-03-21T06:17:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2165.g63e11116.el9.x86_64 31/34 2024-03-21T06:17:26.790 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /sys 2024-03-21T06:17:26.790 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /proc 2024-03-21T06:17:26.790 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /mnt 2024-03-21T06:17:26.790 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /var/tmp 2024-03-21T06:17:26.790 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /home 2024-03-21T06:17:26.790 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /root 2024-03-21T06:17:26.790 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /tmp 2024-03-21T06:17:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:17:27.741 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-radosgw-2:19.0.0-2165.g63e11116.el9.x86_64 32/34 2024-03-21T06:17:27.765 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2165.g63e11116.el9.x86_64 32/34 2024-03-21T06:17:27.765 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T06:17:27.765 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T06:17:27.765 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T06:17:27.765 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T06:17:27.765 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:17:27.767 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-21T06:17:27.861 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-21T06:17:27.861 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T06:17:29.002 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T06:17:29.003 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-base-2:19.0.0-2165.g63e11116.el9.x86_64 1/34 2024-03-21T06:17:29.003 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-common-2:19.0.0-2165.g63e11116.el9.x86_64 2/34 2024-03-21T06:17:29.003 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-radosgw-2:19.0.0-2165.g63e11116.el9.x86_64 3/34 2024-03-21T06:17:29.003 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-selinux-2:19.0.0-2165.g63e11116.el9.x86_64 4/34 2024-03-21T06:17:29.003 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs2-2:19.0.0-2165.g63e11116.el9.x86_64 5/34 2024-03-21T06:17:29.003 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libradosstriper1-2:19.0.0-2165.g63e11116.el9.x86_6 6/34 2024-03-21T06:17:29.003 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librgw2-2:19.0.0-2165.g63e11116.el9.x86_64 7/34 2024-03-21T06:17:29.003 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2165.g63e11116.el9. 8/34 2024-03-21T06:17:29.003 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-common-2:19.0.0-2165.g63e11116.el9.x8 9/34 2024-03-21T06:17:29.003 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cephfs-2:19.0.0-2165.g63e11116.el9.x86_64 10/34 2024-03-21T06:17:29.003 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rados-2:19.0.0-2165.g63e11116.el9.x86_64 11/34 2024-03-21T06:17:29.003 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rbd-2:19.0.0-2165.g63e11116.el9.x86_64 12/34 2024-03-21T06:17:29.004 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rgw-2:19.0.0-2165.g63e11116.el9.x86_64 13/34 2024-03-21T06:17:29.004 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-03-21T06:17:29.004 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-03-21T06:17:29.004 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-03-21T06:17:29.004 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-03-21T06:17:29.004 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-03-21T06:17:29.004 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-03-21T06:17:29.004 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-03-21T06:17:29.004 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-03-21T06:17:29.004 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-03-21T06:17:29.004 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-03-21T06:17:29.004 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-03-21T06:17:29.004 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-03-21T06:17:29.005 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-03-21T06:17:29.006 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-03-21T06:17:29.006 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-03-21T06:17:29.006 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-03-21T06:17:29.006 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-03-21T06:17:29.006 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:19.0.0-2165.g63e11116.el9.x86_64 31/34 2024-03-21T06:17:29.006 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-03-21T06:17:29.006 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:19.0.0-2165.g63e11116.el9.x86_64 33/34 2024-03-21T06:17:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T06:17:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:17:30.070 INFO:teuthology.orchestra.run.smithi096.stdout:Upgraded: 2024-03-21T06:17:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: librados2-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:17:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:17:30.070 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-21T06:17:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T06:17:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:17:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:17:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:17:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:17:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T06:17:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T06:17:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T06:17:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:17:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T06:17:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T06:17:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T06:17:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:17:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T06:17:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:17:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T06:17:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T06:17:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T06:17:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:17:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:17:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:17:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T06:17:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:17:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:17:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:17:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T06:17:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T06:17:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T06:17:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T06:17:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T06:17:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:17:30.073 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:17:30.406 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-test 2024-03-21T06:17:30.961 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:04:55 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:17:31.108 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:17:31.109 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:17:31.109 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-21T06:17:31.109 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:17:31.109 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-21T06:17:31.109 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test x86_64 2:19.0.0-2165.g63e11116.el9 ceph 123 M 2024-03-21T06:17:31.109 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-21T06:17:31.109 INFO:teuthology.orchestra.run.smithi096.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T06:17:31.110 INFO:teuthology.orchestra.run.smithi096.stdout: libasan x86_64 11.4.1-3.el9 appstream 418 k 2024-03-21T06:17:31.110 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite x86_64 2:19.0.0-2165.g63e11116.el9 ceph 158 k 2024-03-21T06:17:31.110 INFO:teuthology.orchestra.run.smithi096.stdout: libubsan x86_64 11.4.1-3.el9 appstream 187 k 2024-03-21T06:17:31.110 INFO:teuthology.orchestra.run.smithi096.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T06:17:31.110 INFO:teuthology.orchestra.run.smithi096.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T06:17:31.110 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T06:17:31.110 INFO:teuthology.orchestra.run.smithi096.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-03-21T06:17:31.110 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:17:31.110 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-21T06:17:31.110 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:17:31.110 INFO:teuthology.orchestra.run.smithi096.stdout:Install 9 Packages 2024-03-21T06:17:31.110 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:17:31.111 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 124 M 2024-03-21T06:17:31.111 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 1.1 G 2024-03-21T06:17:31.111 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-21T06:17:31.840 INFO:teuthology.orchestra.run.smithi096.stdout:(1/9): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.4 MB/s | 217 kB 00:00 2024-03-21T06:17:31.882 INFO:teuthology.orchestra.run.smithi096.stdout:(2/9): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 300 kB 00:00 2024-03-21T06:17:31.907 INFO:teuthology.orchestra.run.smithi096.stdout:(3/9): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.4 MB/s | 63 kB 00:00 2024-03-21T06:17:31.991 INFO:teuthology.orchestra.run.smithi096.stdout:(4/9): libcephsqlite-19.0.0-2165.g63e11116.el9. 518 kB/s | 158 kB 00:00 2024-03-21T06:17:32.099 INFO:teuthology.orchestra.run.smithi096.stdout:(5/9): jq-1.6-16.el9.x86_64.rpm 974 kB/s | 187 kB 00:00 2024-03-21T06:17:32.258 INFO:teuthology.orchestra.run.smithi096.stdout:(6/9): libasan-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 418 kB 00:00 2024-03-21T06:17:32.300 INFO:teuthology.orchestra.run.smithi096.stdout:(7/9): libubsan-11.4.1-3.el9.x86_64.rpm 938 kB/s | 187 kB 00:00 2024-03-21T06:17:32.892 INFO:teuthology.orchestra.run.smithi096.stdout:(8/9): yaml-cpp-0.6.3-5.el9.x86_64.rpm 192 kB/s | 122 kB 00:00 2024-03-21T06:17:37.192 INFO:teuthology.orchestra.run.smithi096.stdout:(9/9): ceph-test-19.0.0-2165.g63e11116.el9.x86_ 22 MB/s | 123 MB 00:05 2024-03-21T06:17:37.196 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:17:37.196 INFO:teuthology.orchestra.run.smithi096.stdout:Total 20 MB/s | 124 MB 00:06 2024-03-21T06:17:37.236 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-21T06:17:37.257 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-21T06:17:37.257 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-21T06:17:38.100 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-21T06:17:38.100 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-21T06:17:39.311 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-21T06:17:39.371 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/9 2024-03-21T06:17:39.408 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libubsan-11.4.1-3.el9.x86_64 2/9 2024-03-21T06:17:39.441 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libasan-11.4.1-3.el9.x86_64 3/9 2024-03-21T06:17:39.485 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 4/9 2024-03-21T06:17:39.527 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 5/9 2024-03-21T06:17:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 6/9 2024-03-21T06:17:39.606 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : jq-1.6-16.el9.x86_64 7/9 2024-03-21T06:17:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephsqlite-2:19.0.0-2165.g63e11116.el9.x86_64 8/9 2024-03-21T06:17:56.471 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2165.g63e11116.el9.x86_64 8/9 2024-03-21T06:17:56.542 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-test-2:19.0.0-2165.g63e11116.el9.x86_64 9/9 2024-03-21T06:18:04.690 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-test-2:19.0.0-2165.g63e11116.el9.x86_64 9/9 2024-03-21T06:18:04.690 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-test-2:19.0.0-2165.g63e11116.el9.x86_64 1/9 2024-03-21T06:18:04.690 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephsqlite-2:19.0.0-2165.g63e11116.el9.x86_64 2/9 2024-03-21T06:18:04.691 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/9 2024-03-21T06:18:04.691 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/9 2024-03-21T06:18:04.691 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/9 2024-03-21T06:18:04.691 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : jq-1.6-16.el9.x86_64 6/9 2024-03-21T06:18:04.691 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libasan-11.4.1-3.el9.x86_64 7/9 2024-03-21T06:18:04.691 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libubsan-11.4.1-3.el9.x86_64 8/9 2024-03-21T06:18:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 9/9 2024-03-21T06:18:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:05.047 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-21T06:18:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:18:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: jq-1.6-16.el9.x86_64 2024-03-21T06:18:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: libasan-11.4.1-3.el9.x86_64 2024-03-21T06:18:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:18:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: libubsan-11.4.1-3.el9.x86_64 2024-03-21T06:18:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T06:18:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T06:18:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T06:18:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-03-21T06:18:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:05.048 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:18:05.338 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph 2024-03-21T06:18:05.896 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:05:30 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:18:06.047 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:18:06.050 INFO:teuthology.orchestra.run.smithi096.stdout:========================================================================================= 2024-03-21T06:18:06.050 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-21T06:18:06.050 INFO:teuthology.orchestra.run.smithi096.stdout:========================================================================================= 2024-03-21T06:18:06.050 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-21T06:18:06.050 INFO:teuthology.orchestra.run.smithi096.stdout: ceph x86_64 2:19.0.0-2165.g63e11116.el9 ceph 6.6 k 2024-03-21T06:18:06.050 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-21T06:18:06.050 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds x86_64 2:19.0.0-2165.g63e11116.el9 ceph 2.2 M 2024-03-21T06:18:06.050 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr x86_64 2:19.0.0-2165.g63e11116.el9 ceph 1.5 M 2024-03-21T06:18:06.050 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 248 k 2024-03-21T06:18:06.051 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon x86_64 2:19.0.0-2165.g63e11116.el9 ceph 4.9 M 2024-03-21T06:18:06.051 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd x86_64 2:19.0.0-2165.g63e11116.el9 ceph 119 M 2024-03-21T06:18:06.051 INFO:teuthology.orchestra.run.smithi096.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T06:18:06.051 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T06:18:06.051 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T06:18:06.051 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T06:18:06.051 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T06:18:06.051 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T06:18:06.051 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T06:18:06.051 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T06:18:06.051 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T06:18:06.051 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T06:18:06.051 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T06:18:06.051 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T06:18:06.052 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T06:18:06.052 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T06:18:06.052 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T06:18:06.052 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T06:18:06.052 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T06:18:06.052 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T06:18:06.052 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T06:18:06.052 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T06:18:06.052 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T06:18:06.052 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T06:18:06.052 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T06:18:06.052 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T06:18:06.052 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T06:18:06.053 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T06:18:06.053 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T06:18:06.053 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T06:18:06.053 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T06:18:06.053 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T06:18:06.053 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T06:18:06.053 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:06.053 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-21T06:18:06.053 INFO:teuthology.orchestra.run.smithi096.stdout:========================================================================================= 2024-03-21T06:18:06.053 INFO:teuthology.orchestra.run.smithi096.stdout:Install 37 Packages 2024-03-21T06:18:06.053 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:06.054 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 133 M 2024-03-21T06:18:06.054 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 1.4 G 2024-03-21T06:18:06.054 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-21T06:18:06.736 INFO:teuthology.orchestra.run.smithi096.stdout:(1/37): ceph-19.0.0-2165.g63e11116.el9.x86_64.r 31 kB/s | 6.6 kB 00:00 2024-03-21T06:18:07.002 INFO:teuthology.orchestra.run.smithi096.stdout:(2/37): ceph-mgr-19.0.0-2165.g63e11116.el9.x86_ 3.1 MB/s | 1.5 MB 00:00 2024-03-21T06:18:07.086 INFO:teuthology.orchestra.run.smithi096.stdout:(3/37): ceph-mds-19.0.0-2165.g63e11116.el9.x86_ 4.0 MB/s | 2.2 MB 00:00 2024-03-21T06:18:07.162 INFO:teuthology.orchestra.run.smithi096.stdout:(4/37): ceph-mgr-modules-core-19.0.0-2165.g63e1 3.2 MB/s | 248 kB 00:00 2024-03-21T06:18:07.287 INFO:teuthology.orchestra.run.smithi096.stdout:(5/37): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-03-21T06:18:07.354 INFO:teuthology.orchestra.run.smithi096.stdout:(6/37): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-03-21T06:18:07.388 INFO:teuthology.orchestra.run.smithi096.stdout:(7/37): python3-mako-1.1.4-6.el9.noarch.rpm 4.2 MB/s | 144 kB 00:00 2024-03-21T06:18:07.412 INFO:teuthology.orchestra.run.smithi096.stdout:(8/37): python3-markupsafe-1.1.1-12.el9.x86_64. 1.2 MB/s | 32 kB 00:00 2024-03-21T06:18:07.438 INFO:teuthology.orchestra.run.smithi096.stdout:(9/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.8 MB/s | 47 kB 00:00 2024-03-21T06:18:07.463 INFO:teuthology.orchestra.run.smithi096.stdout:(10/37): python3-toml-0.10.2-6.el9.noarch.rpm 1.5 MB/s | 39 kB 00:00 2024-03-21T06:18:07.497 INFO:teuthology.orchestra.run.smithi096.stdout:(11/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 7.1 MB/s | 241 kB 00:00 2024-03-21T06:18:07.555 INFO:teuthology.orchestra.run.smithi096.stdout:(12/37): python3-cryptography-36.0.1-4.el9.x86_ 20 MB/s | 1.1 MB 00:00 2024-03-21T06:18:07.631 INFO:teuthology.orchestra.run.smithi096.stdout:(13/37): ceph-mon-19.0.0-2165.g63e11116.el9.x86 5.4 MB/s | 4.9 MB 00:00 2024-03-21T06:18:07.656 INFO:teuthology.orchestra.run.smithi096.stdout:(14/37): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-03-21T06:18:07.681 INFO:teuthology.orchestra.run.smithi096.stdout:(15/37): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-03-21T06:18:07.707 INFO:teuthology.orchestra.run.smithi096.stdout:(16/37): python3-urllib3-1.26.5-5.el9.noarch.rp 3.6 MB/s | 187 kB 00:00 2024-03-21T06:18:07.824 INFO:teuthology.orchestra.run.smithi096.stdout:(17/37): lua-devel-5.4.4-4.el9.x86_64.rpm 192 kB/s | 22 kB 00:00 2024-03-21T06:18:07.857 INFO:teuthology.orchestra.run.smithi096.stdout:(18/37): python3-requests-2.25.1-8.el9.noarch.r 713 kB/s | 125 kB 00:00 2024-03-21T06:18:07.999 INFO:teuthology.orchestra.run.smithi096.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 307 kB/s | 43 kB 00:00 2024-03-21T06:18:08.033 INFO:teuthology.orchestra.run.smithi096.stdout:(20/37): luarocks-3.9.2-1.el9.noarch.rpm 723 kB/s | 151 kB 00:00 2024-03-21T06:18:08.133 INFO:teuthology.orchestra.run.smithi096.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-03-21T06:18:08.159 INFO:teuthology.orchestra.run.smithi096.stdout:(22/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.8 MB/s | 358 kB 00:00 2024-03-21T06:18:08.192 INFO:teuthology.orchestra.run.smithi096.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 182 kB/s | 11 kB 00:00 2024-03-21T06:18:08.217 INFO:teuthology.orchestra.run.smithi096.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 301 kB/s | 18 kB 00:00 2024-03-21T06:18:08.259 INFO:teuthology.orchestra.run.smithi096.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 346 kB/s | 23 kB 00:00 2024-03-21T06:18:08.285 INFO:teuthology.orchestra.run.smithi096.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 290 kB/s | 19 kB 00:00 2024-03-21T06:18:08.318 INFO:teuthology.orchestra.run.smithi096.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 337 kB/s | 20 kB 00:00 2024-03-21T06:18:08.344 INFO:teuthology.orchestra.run.smithi096.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 792 kB/s | 46 kB 00:00 2024-03-21T06:18:08.377 INFO:teuthology.orchestra.run.smithi096.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-03-21T06:18:08.419 INFO:teuthology.orchestra.run.smithi096.stdout:(30/37): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-03-21T06:18:08.445 INFO:teuthology.orchestra.run.smithi096.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 244 kB/s | 16 kB 00:00 2024-03-21T06:18:08.478 INFO:teuthology.orchestra.run.smithi096.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-03-21T06:18:08.495 INFO:teuthology.orchestra.run.smithi096.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 713 kB/s | 36 kB 00:00 2024-03-21T06:18:08.546 INFO:teuthology.orchestra.run.smithi096.stdout:(34/37): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-03-21T06:18:08.605 INFO:teuthology.orchestra.run.smithi096.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 341 kB/s | 20 kB 00:00 2024-03-21T06:18:08.629 INFO:teuthology.orchestra.run.smithi096.stdout:(36/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.1 MB/s | 427 kB 00:00 2024-03-21T06:18:12.838 INFO:teuthology.orchestra.run.smithi096.stdout:(37/37): ceph-osd-19.0.0-2165.g63e11116.el9.x86 20 MB/s | 119 MB 00:05 2024-03-21T06:18:12.841 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:18:12.842 INFO:teuthology.orchestra.run.smithi096.stdout:Total 20 MB/s | 133 MB 00:06 2024-03-21T06:18:12.867 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-21T06:18:12.908 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-21T06:18:12.909 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-21T06:18:13.913 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-21T06:18:13.913 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-21T06:18:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-21T06:18:15.865 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-03-21T06:18:15.897 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-03-21T06:18:15.925 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-03-21T06:18:15.986 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-03-21T06:18:16.032 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/37 2024-03-21T06:18:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-03-21T06:18:16.156 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-03-21T06:18:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-03-21T06:18:16.285 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/37 2024-03-21T06:18:16.355 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-03-21T06:18:16.402 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/37 2024-03-21T06:18:16.441 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/37 2024-03-21T06:18:16.498 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/37 2024-03-21T06:18:16.576 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/37 2024-03-21T06:18:16.764 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/37 2024-03-21T06:18:16.804 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/37 2024-03-21T06:18:16.851 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/37 2024-03-21T06:18:16.885 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/37 2024-03-21T06:18:16.916 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/37 2024-03-21T06:18:16.991 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/37 2024-03-21T06:18:17.026 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/37 2024-03-21T06:18:17.059 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/37 2024-03-21T06:18:17.100 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/37 2024-03-21T06:18:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/37 2024-03-21T06:18:17.169 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/37 2024-03-21T06:18:17.257 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/37 2024-03-21T06:18:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/37 2024-03-21T06:18:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/37 2024-03-21T06:18:17.508 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/37 2024-03-21T06:18:17.625 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/37 2024-03-21T06:18:17.801 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2165.g63e11116.el9. 31/37 2024-03-21T06:18:17.831 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-2:19.0.0-2165.g63e11116.el9.x86_64 32/37 2024-03-21T06:18:17.856 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2165.g63e11116.el9.x86_64 32/37 2024-03-21T06:18:17.856 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T06:18:17.856 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T06:18:17.856 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T06:18:17.856 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T06:18:17.856 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:36.769 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 33/37 2024-03-21T06:18:36.879 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-osd-2:19.0.0-2165.g63e11116.el9.x86_64 34/37 2024-03-21T06:18:36.905 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-osd-2:19.0.0-2165.g63e11116.el9.x86_64 34/37 2024-03-21T06:18:36.905 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T06:18:36.905 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T06:18:36.905 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T06:18:36.905 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T06:18:36.905 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:37.380 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mon-2:19.0.0-2165.g63e11116.el9.x86_64 35/37 2024-03-21T06:18:37.405 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mon-2:19.0.0-2165.g63e11116.el9.x86_64 35/37 2024-03-21T06:18:37.405 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T06:18:37.405 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T06:18:37.405 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T06:18:37.405 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T06:18:37.406 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:37.639 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mds-2:19.0.0-2165.g63e11116.el9.x86_64 36/37 2024-03-21T06:18:37.660 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mds-2:19.0.0-2165.g63e11116.el9.x86_64 36/37 2024-03-21T06:18:37.661 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T06:18:37.661 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T06:18:37.661 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T06:18:37.661 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T06:18:37.661 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:37.713 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-2:19.0.0-2165.g63e11116.el9.x86_64 37/37 2024-03-21T06:18:47.685 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-2:19.0.0-2165.g63e11116.el9.x86_64 37/37 2024-03-21T06:18:47.685 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-2:19.0.0-2165.g63e11116.el9.x86_64 1/37 2024-03-21T06:18:47.685 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mds-2:19.0.0-2165.g63e11116.el9.x86_64 2/37 2024-03-21T06:18:47.686 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-2:19.0.0-2165.g63e11116.el9.x86_64 3/37 2024-03-21T06:18:47.686 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mon-2:19.0.0-2165.g63e11116.el9.x86_64 4/37 2024-03-21T06:18:47.686 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-osd-2:19.0.0-2165.g63e11116.el9.x86_64 5/37 2024-03-21T06:18:47.687 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2165.g63e11116.el9. 6/37 2024-03-21T06:18:47.687 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/37 2024-03-21T06:18:47.687 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 8/37 2024-03-21T06:18:47.687 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 9/37 2024-03-21T06:18:47.687 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 10/37 2024-03-21T06:18:47.687 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 11/37 2024-03-21T06:18:47.687 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 12/37 2024-03-21T06:18:47.687 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 13/37 2024-03-21T06:18:47.688 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 14/37 2024-03-21T06:18:47.688 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ply-3.11-14.el9.noarch 15/37 2024-03-21T06:18:47.688 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 16/37 2024-03-21T06:18:47.688 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 17/37 2024-03-21T06:18:47.689 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 18/37 2024-03-21T06:18:47.689 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-03-21T06:18:47.689 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 20/37 2024-03-21T06:18:47.689 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-03-21T06:18:47.689 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-03-21T06:18:47.689 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-03-21T06:18:47.689 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-03-21T06:18:47.690 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-03-21T06:18:47.690 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-03-21T06:18:47.690 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-03-21T06:18:47.690 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-03-21T06:18:47.690 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-03-21T06:18:47.690 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-03-21T06:18:47.690 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-03-21T06:18:47.691 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-03-21T06:18:47.691 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-03-21T06:18:47.691 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-03-21T06:18:47.691 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 35/37 2024-03-21T06:18:47.691 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-03-21T06:18:48.517 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-03-21T06:18:48.518 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:48.518 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-21T06:18:48.518 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:18:48.518 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:18:48.518 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:18:48.518 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:18:48.518 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:18:48.518 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:18:48.518 INFO:teuthology.orchestra.run.smithi096.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T06:18:48.518 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T06:18:48.518 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T06:18:48.518 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T06:18:48.519 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T06:18:48.519 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T06:18:48.519 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T06:18:48.519 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T06:18:48.519 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T06:18:48.519 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T06:18:48.519 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T06:18:48.519 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T06:18:48.519 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T06:18:48.519 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T06:18:48.519 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T06:18:48.520 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T06:18:48.520 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T06:18:48.520 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T06:18:48.520 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T06:18:48.520 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T06:18:48.520 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T06:18:48.520 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T06:18:48.520 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T06:18:48.520 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T06:18:48.520 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T06:18:48.520 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T06:18:48.520 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T06:18:48.520 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T06:18:48.521 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T06:18:48.521 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T06:18:48.521 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T06:18:48.521 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:48.521 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:18:48.770 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-base 2024-03-21T06:18:49.324 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:06:14 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:18:49.421 INFO:teuthology.orchestra.run.smithi096.stdout:Package ceph-base-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:18:49.474 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:18:49.475 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-21T06:18:49.475 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:18:49.540 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install cephadm 2024-03-21T06:18:50.096 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:06:15 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:18:50.246 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:18:50.247 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:18:50.247 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-21T06:18:50.247 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:18:50.247 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-21T06:18:50.247 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 748 k 2024-03-21T06:18:50.247 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:50.248 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-21T06:18:50.248 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:18:50.248 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-03-21T06:18:50.248 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:50.248 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 748 k 2024-03-21T06:18:50.249 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 755 k 2024-03-21T06:18:50.249 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-21T06:18:50.536 INFO:teuthology.orchestra.run.smithi096.stdout:cephadm-19.0.0-2165.g63e11116.el9.noarch.rpm 2.5 MB/s | 748 kB 00:00 2024-03-21T06:18:50.536 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:18:50.537 INFO:teuthology.orchestra.run.smithi096.stdout:Total 2.5 MB/s | 748 kB 00:00 2024-03-21T06:18:50.537 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-21T06:18:50.542 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-21T06:18:50.542 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-21T06:18:50.548 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-21T06:18:50.548 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-21T06:18:50.623 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-21T06:18:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:19.0.0-2165.g63e11116.el9.noarch 1/1 2024-03-21T06:18:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephadm-2:19.0.0-2165.g63e11116.el9.noarch 1/1 2024-03-21T06:18:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:19.0.0-2165.g63e11116.el9.noarch 1/1 2024-03-21T06:18:51.891 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephadm-2:19.0.0-2165.g63e11116.el9.noarch 1/1 2024-03-21T06:18:51.892 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:51.892 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-21T06:18:51.892 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:18:51.892 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:51.892 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:18:52.084 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-immutable-object-cache 2024-03-21T06:18:52.651 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:06:17 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:18:52.806 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2024-03-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2165.g63e11116.el9 ceph 147 k 2024-03-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:52.807 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-03-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 147 k 2024-03-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 481 k 2024-03-21T06:18:52.808 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-21T06:18:53.035 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-immutable-object-cache-19.0.0-2165.g63e111 650 kB/s | 147 kB 00:00 2024-03-21T06:18:53.035 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:18:53.035 INFO:teuthology.orchestra.run.smithi096.stdout:Total 646 kB/s | 147 kB 00:00 2024-03-21T06:18:53.035 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-21T06:18:53.043 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-21T06:18:53.043 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-21T06:18:53.095 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-21T06:18:53.095 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-21T06:18:53.237 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-21T06:18:53.294 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2165.g63e11116. 1/1 2024-03-21T06:18:53.318 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2165.g63e11116. 1/1 2024-03-21T06:18:53.318 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T06:18:53.318 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T06:18:53.318 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:54.272 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2165.g63e11116. 1/1 2024-03-21T06:18:54.272 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:54.272 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-21T06:18:54.272 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:18:54.272 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:54.272 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:18:54.488 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr 2024-03-21T06:18:55.056 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:06:20 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:18:55.153 INFO:teuthology.orchestra.run.smithi096.stdout:Package ceph-mgr-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:18:55.204 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:18:55.205 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-21T06:18:55.205 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:18:55.261 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-dashboard 2024-03-21T06:18:55.820 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:06:20 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:18:55.970 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:18:55.970 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================== 2024-03-21T06:18:55.971 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-21T06:18:55.971 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================== 2024-03-21T06:18:55.971 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-21T06:18:55.971 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 1.8 M 2024-03-21T06:18:55.971 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-21T06:18:55.971 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T06:18:55.971 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 34 k 2024-03-21T06:18:55.971 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 16 k 2024-03-21T06:18:55.971 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T06:18:55.971 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T06:18:55.971 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T06:18:55.971 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T06:18:55.971 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T06:18:55.971 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T06:18:55.972 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T06:18:55.972 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:55.972 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-21T06:18:55.972 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================== 2024-03-21T06:18:55.972 INFO:teuthology.orchestra.run.smithi096.stdout:Install 11 Packages 2024-03-21T06:18:55.972 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:18:55.972 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 5.9 M 2024-03-21T06:18:55.973 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 23 M 2024-03-21T06:18:55.973 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-21T06:18:56.364 INFO:teuthology.orchestra.run.smithi096.stdout:(1/11): ceph-prometheus-alerts-19.0.0-2165.g63e 76 kB/s | 16 kB 00:00 2024-03-21T06:18:56.406 INFO:teuthology.orchestra.run.smithi096.stdout:(2/11): ceph-grafana-dashboards-19.0.0-2165.g63 132 kB/s | 34 kB 00:00 2024-03-21T06:18:56.523 INFO:teuthology.orchestra.run.smithi096.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.5 MB/s | 238 kB 00:00 2024-03-21T06:18:56.565 INFO:teuthology.orchestra.run.smithi096.stdout:(4/11): ceph-mgr-dashboard-19.0.0-2165.g63e1111 4.3 MB/s | 1.8 MB 00:00 2024-03-21T06:18:56.923 INFO:teuthology.orchestra.run.smithi096.stdout:(5/11): grpc-data-1.46.7-10.el9.noarch.rpm 54 kB/s | 19 kB 00:00 2024-03-21T06:18:57.307 INFO:teuthology.orchestra.run.smithi096.stdout:(6/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 699 kB/s | 548 kB 00:00 2024-03-21T06:18:57.432 INFO:teuthology.orchestra.run.smithi096.stdout:(7/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.1 MB/s | 144 kB 00:00 2024-03-21T06:18:57.557 INFO:teuthology.orchestra.run.smithi096.stdout:(8/11): python3-repoze-lru-0.7-16.el9.noarch.rp 246 kB/s | 31 kB 00:00 2024-03-21T06:18:57.683 INFO:teuthology.orchestra.run.smithi096.stdout:(9/11): python3-routes-2.5.1-5.el9.noarch.rpm 1.5 MB/s | 188 kB 00:00 2024-03-21T06:18:57.741 INFO:teuthology.orchestra.run.smithi096.stdout:(10/11): python3-grpcio-1.46.7-10.el9.x86_64.rp 2.5 MB/s | 2.0 MB 00:00 2024-03-21T06:18:59.333 INFO:teuthology.orchestra.run.smithi096.stdout:(11/11): protobuf-compiler-3.14.0-13.el9.x86_64 295 kB/s | 863 kB 00:02 2024-03-21T06:18:59.338 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:18:59.338 INFO:teuthology.orchestra.run.smithi096.stdout:Total 1.7 MB/s | 5.9 MB 00:03 2024-03-21T06:18:59.358 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-21T06:18:59.377 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-21T06:18:59.377 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-21T06:18:59.550 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-21T06:18:59.550 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-21T06:19:00.039 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-21T06:19:00.131 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-03-21T06:19:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-03-21T06:19:00.268 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-03-21T06:19:00.395 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-03-21T06:19:00.440 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-03-21T06:19:00.523 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-03-21T06:19:00.567 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-03-21T06:19:00.595 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-03-21T06:19:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2165.g63e11116.el9 9/11 2024-03-21T06:19:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2165.g63e11116.el 10/11 2024-03-21T06:19:00.902 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2165.g63e11116.el9.noa 11/11 2024-03-21T06:19:01.662 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2165.g63e11116.el9.noa 11/11 2024-03-21T06:19:01.662 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2165.g63e11116.el 1/11 2024-03-21T06:19:01.662 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2165.g63e11116.el9.noa 2/11 2024-03-21T06:19:01.662 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2165.g63e11116.el9 3/11 2024-03-21T06:19:01.662 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-03-21T06:19:01.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-03-21T06:19:01.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-03-21T06:19:01.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-03-21T06:19:01.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-03-21T06:19:01.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-03-21T06:19:01.663 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-03-21T06:19:01.990 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-03-21T06:19:01.990 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:01.990 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-21T06:19:01.990 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T06:19:01.990 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:19:01.990 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:19:01.990 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:19:01.990 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T06:19:01.990 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T06:19:01.990 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T06:19:01.990 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T06:19:01.991 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T06:19:01.991 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T06:19:01.991 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T06:19:01.991 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:01.991 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:02.272 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-21T06:19:02.835 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:06:27 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:19:02.984 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:19:02.985 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================================ 2024-03-21T06:19:02.985 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-21T06:19:02.985 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================================ 2024-03-21T06:19:02.985 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-21T06:19:02.986 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 7.4 M 2024-03-21T06:19:02.986 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-21T06:19:02.986 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T06:19:02.986 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T06:19:02.986 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T06:19:02.986 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T06:19:02.986 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T06:19:02.986 INFO:teuthology.orchestra.run.smithi096.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T06:19:02.986 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T06:19:02.986 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T06:19:02.986 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T06:19:02.986 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T06:19:02.986 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T06:19:02.986 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:02.987 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-21T06:19:02.987 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================================ 2024-03-21T06:19:02.987 INFO:teuthology.orchestra.run.smithi096.stdout:Install 12 Packages 2024-03-21T06:19:02.987 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:02.987 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 37 M 2024-03-21T06:19:02.987 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 209 M 2024-03-21T06:19:02.987 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-21T06:19:03.215 INFO:teuthology.orchestra.run.smithi096.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 222 kB/s | 32 kB 00:00 2024-03-21T06:19:03.248 INFO:teuthology.orchestra.run.smithi096.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 534 kB/s | 17 kB 00:00 2024-03-21T06:19:03.290 INFO:teuthology.orchestra.run.smithi096.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 840 kB/s | 34 kB 00:00 2024-03-21T06:19:03.340 INFO:teuthology.orchestra.run.smithi096.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-03-21T06:19:03.624 INFO:teuthology.orchestra.run.smithi096.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2024-03-21T06:19:03.691 INFO:teuthology.orchestra.run.smithi096.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 4.8 MB 00:00 2024-03-21T06:19:03.716 INFO:teuthology.orchestra.run.smithi096.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.3 MB/s | 217 kB 00:00 2024-03-21T06:19:04.208 INFO:teuthology.orchestra.run.smithi096.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-03-21T06:19:05.000 INFO:teuthology.orchestra.run.smithi096.stdout:(9/12): ceph-mgr-diskprediction-local-19.0.0-21 3.8 MB/s | 7.4 MB 00:01 2024-03-21T06:19:05.092 INFO:teuthology.orchestra.run.smithi096.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 221 kB/s | 195 kB 00:00 2024-03-21T06:19:05.217 INFO:teuthology.orchestra.run.smithi096.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-03-21T06:19:05.567 INFO:teuthology.orchestra.run.smithi096.stdout:(12/12): libgfortran-11.4.1-3.el9.x86_64.rpm 434 kB/s | 803 kB 00:01 2024-03-21T06:19:05.571 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:19:05.571 INFO:teuthology.orchestra.run.smithi096.stdout:Total 14 MB/s | 37 MB 00:02 2024-03-21T06:19:05.615 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-21T06:19:05.641 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-21T06:19:05.641 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-21T06:19:06.005 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-21T06:19:06.005 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-21T06:19:06.726 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-21T06:19:06.790 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-21T06:19:06.821 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-21T06:19:06.910 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-21T06:19:06.958 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-21T06:19:07.137 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-21T06:19:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-21T06:19:07.242 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-21T06:19:07.900 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-21T06:19:08.013 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-21T06:19:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-21T06:19:10.310 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-21T06:19:10.332 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2165.g63e11 12/12 2024-03-21T06:19:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2165.g63e11 12/12 2024-03-21T06:19:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2165.g63e11 1/12 2024-03-21T06:19:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-21T06:19:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-21T06:19:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-21T06:19:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-21T06:19:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-21T06:19:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-21T06:19:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-21T06:19:12.985 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-21T06:19:12.985 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-21T06:19:12.985 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-21T06:19:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-21T06:19:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:13.424 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-21T06:19:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:19:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T06:19:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T06:19:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T06:19:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T06:19:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T06:19:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T06:19:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T06:19:13.424 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T06:19:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T06:19:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T06:19:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T06:19:13.425 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:13.425 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:13.866 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-rook 2024-03-21T06:19:14.437 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:06:39 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:19:14.588 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:19:14.589 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================== 2024-03-21T06:19:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-21T06:19:14.589 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================== 2024-03-21T06:19:14.589 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-21T06:19:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 49 k 2024-03-21T06:19:14.589 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-21T06:19:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T06:19:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T06:19:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T06:19:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T06:19:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T06:19:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T06:19:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T06:19:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T06:19:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T06:19:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T06:19:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T06:19:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T06:19:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:14.591 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-21T06:19:14.591 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================== 2024-03-21T06:19:14.591 INFO:teuthology.orchestra.run.smithi096.stdout:Install 13 Packages 2024-03-21T06:19:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:14.591 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 2.0 M 2024-03-21T06:19:14.591 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 26 M 2024-03-21T06:19:14.592 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-21T06:19:14.758 INFO:teuthology.orchestra.run.smithi096.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 214 kB/s | 24 kB 00:00 2024-03-21T06:19:14.784 INFO:teuthology.orchestra.run.smithi096.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 130 kB/s | 18 kB 00:00 2024-03-21T06:19:14.826 INFO:teuthology.orchestra.run.smithi096.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 174 kB 00:00 2024-03-21T06:19:14.851 INFO:teuthology.orchestra.run.smithi096.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-21T06:19:14.884 INFO:teuthology.orchestra.run.smithi096.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.5 MB/s | 211 kB 00:00 2024-03-21T06:19:14.910 INFO:teuthology.orchestra.run.smithi096.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 731 kB/s | 43 kB 00:00 2024-03-21T06:19:14.935 INFO:teuthology.orchestra.run.smithi096.stdout:(7/13): ceph-mgr-rook-19.0.0-2165.g63e11116.el9 169 kB/s | 49 kB 00:00 2024-03-21T06:19:14.993 INFO:teuthology.orchestra.run.smithi096.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 179 kB/s | 15 kB 00:00 2024-03-21T06:19:15.019 INFO:teuthology.orchestra.run.smithi096.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 240 kB/s | 32 kB 00:00 2024-03-21T06:19:15.071 INFO:teuthology.orchestra.run.smithi096.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 1.5 MB/s | 203 kB 00:00 2024-03-21T06:19:15.097 INFO:teuthology.orchestra.run.smithi096.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 762 kB/s | 59 kB 00:00 2024-03-21T06:19:15.122 INFO:teuthology.orchestra.run.smithi096.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-03-21T06:19:15.297 INFO:teuthology.orchestra.run.smithi096.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.4 MB/s | 1.0 MB 00:00 2024-03-21T06:19:15.299 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:19:15.299 INFO:teuthology.orchestra.run.smithi096.stdout:Total 2.9 MB/s | 2.0 MB 00:00 2024-03-21T06:19:15.300 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-21T06:19:15.313 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-21T06:19:15.313 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-21T06:19:15.403 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-21T06:19:15.403 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-21T06:19:15.919 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-21T06:19:16.087 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-21T06:19:16.132 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-21T06:19:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-21T06:19:16.224 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-21T06:19:16.266 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-21T06:19:16.359 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-21T06:19:16.459 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-21T06:19:16.508 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-21T06:19:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-21T06:19:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-21T06:19:17.307 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-21T06:19:17.355 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-21T06:19:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-rook-2:19.0.0-2165.g63e11116.el9.noarch 13/13 2024-03-21T06:19:18.121 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2165.g63e11116.el9.noarch 13/13 2024-03-21T06:19:18.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2165.g63e11116.el9.noarch 1/13 2024-03-21T06:19:18.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-21T06:19:18.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-21T06:19:18.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-21T06:19:18.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-21T06:19:18.121 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-21T06:19:18.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-21T06:19:18.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-21T06:19:18.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-21T06:19:18.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-21T06:19:18.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-21T06:19:18.122 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-21T06:19:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-21T06:19:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:18.472 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-21T06:19:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:19:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T06:19:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T06:19:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T06:19:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T06:19:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T06:19:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T06:19:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T06:19:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T06:19:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T06:19:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T06:19:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T06:19:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T06:19:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:18.473 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:18.719 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-cephadm 2024-03-21T06:19:19.278 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:06:44 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:19:19.426 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:19:19.427 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================== 2024-03-21T06:19:19.427 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-21T06:19:19.427 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================== 2024-03-21T06:19:19.427 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-21T06:19:19.427 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 139 k 2024-03-21T06:19:19.428 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-21T06:19:19.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T06:19:19.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T06:19:19.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T06:19:19.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T06:19:19.428 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T06:19:19.428 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:19.428 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-21T06:19:19.428 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================== 2024-03-21T06:19:19.428 INFO:teuthology.orchestra.run.smithi096.stdout:Install 6 Packages 2024-03-21T06:19:19.428 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:19.429 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 6.8 M 2024-03-21T06:19:19.429 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 33 M 2024-03-21T06:19:19.429 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-21T06:19:19.908 INFO:teuthology.orchestra.run.smithi096.stdout:(1/6): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2024-03-21T06:19:19.966 INFO:teuthology.orchestra.run.smithi096.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 784 kB/s | 248 kB 00:00 2024-03-21T06:19:19.992 INFO:teuthology.orchestra.run.smithi096.stdout:(3/6): ceph-mgr-cephadm-19.0.0-2165.g63e11116.e 406 kB/s | 139 kB 00:00 2024-03-21T06:19:20.433 INFO:teuthology.orchestra.run.smithi096.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 123 kB/s | 58 kB 00:00 2024-03-21T06:19:20.475 INFO:teuthology.orchestra.run.smithi096.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 111 kB/s | 51 kB 00:00 2024-03-21T06:19:20.725 INFO:teuthology.orchestra.run.smithi096.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 671 kB/s | 548 kB 00:00 2024-03-21T06:19:20.730 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:19:20.730 INFO:teuthology.orchestra.run.smithi096.stdout:Total 5.2 MB/s | 6.8 MB 00:01 2024-03-21T06:19:20.743 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-21T06:19:20.754 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-21T06:19:20.754 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-21T06:19:20.842 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-21T06:19:20.843 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-21T06:19:21.143 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-21T06:19:21.343 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-21T06:19:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-21T06:19:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-21T06:19:22.022 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-21T06:19:22.084 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-21T06:19:22.106 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2165.g63e11116.el9.noarch 6/6 2024-03-21T06:19:22.866 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2165.g63e11116.el9.noarch 6/6 2024-03-21T06:19:22.867 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2165.g63e11116.el9.noarch 1/6 2024-03-21T06:19:22.867 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-21T06:19:22.867 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-21T06:19:22.867 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-21T06:19:22.867 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-21T06:19:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-21T06:19:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:23.171 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-21T06:19:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:19:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T06:19:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T06:19:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T06:19:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T06:19:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T06:19:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:23.171 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:23.423 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-fuse 2024-03-21T06:19:23.982 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:06:48 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:19:24.132 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:19:24.133 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-21T06:19:24.133 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:24.133 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-21T06:19:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse x86_64 2:19.0.0-2165.g63e11116.el9 ceph 861 k 2024-03-21T06:19:24.133 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-21T06:19:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T06:19:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:24.133 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-21T06:19:24.133 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:24.133 INFO:teuthology.orchestra.run.smithi096.stdout:Install 2 Packages 2024-03-21T06:19:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:24.134 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 940 k 2024-03-21T06:19:24.134 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 3.1 M 2024-03-21T06:19:24.134 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-21T06:19:24.261 INFO:teuthology.orchestra.run.smithi096.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 621 kB/s | 79 kB 00:00 2024-03-21T06:19:24.461 INFO:teuthology.orchestra.run.smithi096.stdout:(2/2): ceph-fuse-19.0.0-2165.g63e11116.el9.x86_ 2.6 MB/s | 861 kB 00:00 2024-03-21T06:19:24.462 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:19:24.462 INFO:teuthology.orchestra.run.smithi096.stdout:Total 2.8 MB/s | 940 kB 00:00 2024-03-21T06:19:24.463 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-21T06:19:24.483 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-21T06:19:24.483 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-21T06:19:24.555 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-21T06:19:24.556 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-21T06:19:24.730 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-21T06:19:24.872 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-21T06:19:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-fuse-2:19.0.0-2165.g63e11116.el9.x86_64 2/2 2024-03-21T06:19:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2165.g63e11116.el9.x86_64 2/2 2024-03-21T06:19:25.635 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-fuse-2:19.0.0-2165.g63e11116.el9.x86_64 1/2 2024-03-21T06:19:25.926 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-21T06:19:25.926 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:25.926 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-21T06:19:25.926 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse-2:19.0.0-2165.g63e11116.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-21T06:19:25.926 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:25.927 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:26.206 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-volume 2024-03-21T06:19:26.783 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:06:51 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:19:26.938 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:19:26.939 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-21T06:19:26.939 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:26.939 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-21T06:19:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume noarch 2:19.0.0-2165.g63e11116.el9 ceph-noarch 264 k 2024-03-21T06:19:26.940 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-21T06:19:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T06:19:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:26.940 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-21T06:19:26.940 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:26.940 INFO:teuthology.orchestra.run.smithi096.stdout:Install 2 Packages 2024-03-21T06:19:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:26.940 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 332 k 2024-03-21T06:19:26.940 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 1.5 M 2024-03-21T06:19:26.940 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-21T06:19:27.092 INFO:teuthology.orchestra.run.smithi096.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 454 kB/s | 69 kB 00:00 2024-03-21T06:19:27.259 INFO:teuthology.orchestra.run.smithi096.stdout:(2/2): ceph-volume-19.0.0-2165.g63e11116.el9.no 829 kB/s | 264 kB 00:00 2024-03-21T06:19:27.260 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:19:27.260 INFO:teuthology.orchestra.run.smithi096.stdout:Total 1.0 MB/s | 332 kB 00:00 2024-03-21T06:19:27.260 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-21T06:19:27.268 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-21T06:19:27.269 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-21T06:19:27.307 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-21T06:19:27.307 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-21T06:19:27.524 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-21T06:19:27.708 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-03-21T06:19:27.753 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-volume-2:19.0.0-2165.g63e11116.el9.noarch 2/2 2024-03-21T06:19:27.768 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-volume-2:19.0.0-2165.g63e11116.el9.noarch 2/2 2024-03-21T06:19:27.769 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T06:19:27.769 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T06:19:27.769 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-volume-2:19.0.0-2165.g63e11116.el9.noarch 1/2 2024-03-21T06:19:28.828 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-03-21T06:19:28.828 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:28.828 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-21T06:19:28.828 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume-2:19.0.0-2165.g63e11116.el9.noarch 2024-03-21T06:19:28.829 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T06:19:28.829 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:28.829 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:29.136 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install librados-devel 2024-03-21T06:19:29.710 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:06:54 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:19:29.860 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:19:29.860 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2024-03-21T06:19:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-21T06:19:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel x86_64 2:19.0.0-2165.g63e11116.el9 ceph 127 k 2024-03-21T06:19:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-21T06:19:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-03-21T06:19:29.861 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 127 k 2024-03-21T06:19:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 457 k 2024-03-21T06:19:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-21T06:19:30.132 INFO:teuthology.orchestra.run.smithi096.stdout:librados-devel-19.0.0-2165.g63e11116.el9.x86_64 469 kB/s | 127 kB 00:00 2024-03-21T06:19:30.132 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:19:30.132 INFO:teuthology.orchestra.run.smithi096.stdout:Total 467 kB/s | 127 kB 00:00 2024-03-21T06:19:30.133 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-21T06:19:30.138 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-21T06:19:30.139 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-21T06:19:30.195 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-21T06:19:30.195 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-21T06:19:30.330 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-21T06:19:30.447 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librados-devel-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:19:30.898 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados-devel-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:19:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados-devel-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:19:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:31.176 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-21T06:19:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:19:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:31.176 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:31.412 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install libcephfs2 2024-03-21T06:19:31.989 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:06:56 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:19:32.089 INFO:teuthology.orchestra.run.smithi096.stdout:Package libcephfs2-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:19:32.141 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:19:32.141 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-21T06:19:32.142 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:32.199 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install libcephfs-devel 2024-03-21T06:19:32.758 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:06:57 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:19:32.908 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:19:32.909 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:32.909 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2024-03-21T06:19:32.909 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:32.909 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-21T06:19:32.909 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel x86_64 2:19.0.0-2165.g63e11116.el9 ceph 31 k 2024-03-21T06:19:32.909 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:32.910 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-21T06:19:32.910 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:32.910 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-03-21T06:19:32.910 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:32.910 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 31 k 2024-03-21T06:19:32.910 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 140 k 2024-03-21T06:19:32.910 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-21T06:19:33.061 INFO:teuthology.orchestra.run.smithi096.stdout:libcephfs-devel-19.0.0-2165.g63e11116.el9.x86_6 207 kB/s | 31 kB 00:00 2024-03-21T06:19:33.061 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:19:33.062 INFO:teuthology.orchestra.run.smithi096.stdout:Total 205 kB/s | 31 kB 00:00 2024-03-21T06:19:33.062 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-21T06:19:33.065 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-21T06:19:33.065 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-21T06:19:33.093 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-21T06:19:33.093 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-21T06:19:33.190 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-21T06:19:33.293 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs-devel-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:19:33.677 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:19:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs-devel-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:19:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:33.924 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-21T06:19:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:19:33.924 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:33.924 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:34.109 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install librados2 2024-03-21T06:19:34.668 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:06:59 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:19:34.766 INFO:teuthology.orchestra.run.smithi096.stdout:Package librados2-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:19:34.816 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:19:34.818 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-21T06:19:34.818 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:34.880 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install librbd1 2024-03-21T06:19:35.442 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:07:00 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:19:35.538 INFO:teuthology.orchestra.run.smithi096.stdout:Package librbd1-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:19:35.590 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:19:35.591 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-21T06:19:35.591 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:35.647 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-rados 2024-03-21T06:19:36.208 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:07:01 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:19:36.305 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-rados-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:19:36.357 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:19:36.358 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-21T06:19:36.358 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:36.414 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-rgw 2024-03-21T06:19:36.982 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:07:01 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:19:37.081 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-rgw-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:19:37.131 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:19:37.132 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-21T06:19:37.132 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:37.190 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-cephfs 2024-03-21T06:19:37.752 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:07:02 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:19:37.850 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-cephfs-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:19:37.903 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:19:37.904 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-21T06:19:37.904 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:37.958 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-rbd 2024-03-21T06:19:38.520 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:07:03 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:19:38.618 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-rbd-2:19.0.0-2165.g63e11116.el9.x86_64 is already installed. 2024-03-21T06:19:38.671 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:19:38.751 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-03-21T06:19:38.751 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:38.752 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install rbd-fuse 2024-03-21T06:19:39.308 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:07:04 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:19:39.457 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:19:39.458 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:39.458 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-03-21T06:19:39.458 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:39.458 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-21T06:19:39.458 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse x86_64 2:19.0.0-2165.g63e11116.el9 ceph 86 k 2024-03-21T06:19:39.458 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:39.458 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-21T06:19:39.458 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:39.458 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-03-21T06:19:39.458 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:39.459 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 86 k 2024-03-21T06:19:39.459 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 256 k 2024-03-21T06:19:39.459 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-21T06:19:39.672 INFO:teuthology.orchestra.run.smithi096.stdout:rbd-fuse-19.0.0-2165.g63e11116.el9.x86_64.rpm 405 kB/s | 86 kB 00:00 2024-03-21T06:19:39.672 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:19:39.673 INFO:teuthology.orchestra.run.smithi096.stdout:Total 403 kB/s | 86 kB 00:00 2024-03-21T06:19:39.673 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-21T06:19:39.679 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-21T06:19:39.679 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-21T06:19:39.734 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-21T06:19:39.734 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-21T06:19:39.872 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-21T06:19:39.979 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-fuse-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:19:40.392 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:19:40.691 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-fuse-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:19:40.692 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:40.692 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-21T06:19:40.692 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:19:40.692 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:40.692 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:40.931 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install rbd-mirror 2024-03-21T06:19:41.514 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:07:06 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:19:41.667 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:19:41.668 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2024-03-21T06:19:41.668 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:41.668 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-21T06:19:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror x86_64 2:19.0.0-2165.g63e11116.el9 ceph 3.0 M 2024-03-21T06:19:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:41.668 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-21T06:19:41.668 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:41.668 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-03-21T06:19:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:41.668 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 3.0 M 2024-03-21T06:19:41.668 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 14 M 2024-03-21T06:19:41.668 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-21T06:19:42.101 INFO:teuthology.orchestra.run.smithi096.stdout:rbd-mirror-19.0.0-2165.g63e11116.el9.x86_64.rpm 7.0 MB/s | 3.0 MB 00:00 2024-03-21T06:19:42.102 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:19:42.102 INFO:teuthology.orchestra.run.smithi096.stdout:Total 7.0 MB/s | 3.0 MB 00:00 2024-03-21T06:19:42.102 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-21T06:19:42.111 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-21T06:19:42.111 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-21T06:19:42.184 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-21T06:19:42.184 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-21T06:19:42.624 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-21T06:19:42.686 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-mirror-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:19:42.711 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:19:42.711 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T06:19:42.711 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T06:19:42.711 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T06:19:42.711 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T06:19:42.711 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-mirror-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:19:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:43.784 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-21T06:19:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:19:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:43.784 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:44.051 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install rbd-nbd 2024-03-21T06:19:44.620 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:07:09 ago on Thu 21 Mar 2024 06:12:35 AM UTC. 2024-03-21T06:19:44.769 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-21T06:19:44.769 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:44.770 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-03-21T06:19:44.770 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:44.770 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-21T06:19:44.770 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd x86_64 2:19.0.0-2165.g63e11116.el9 ceph 169 k 2024-03-21T06:19:44.770 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:44.770 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-21T06:19:44.770 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-21T06:19:44.770 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-03-21T06:19:44.770 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:44.770 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 169 k 2024-03-21T06:19:44.770 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 541 k 2024-03-21T06:19:44.770 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-21T06:19:44.981 INFO:teuthology.orchestra.run.smithi096.stdout:rbd-nbd-19.0.0-2165.g63e11116.el9.x86_64.rpm 804 kB/s | 169 kB 00:00 2024-03-21T06:19:44.981 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-21T06:19:44.982 INFO:teuthology.orchestra.run.smithi096.stdout:Total 798 kB/s | 169 kB 00:00 2024-03-21T06:19:44.982 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-21T06:19:44.989 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-21T06:19:44.989 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-21T06:19:45.044 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-21T06:19:45.045 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-21T06:19:45.200 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-21T06:19:45.306 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-nbd-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:19:45.701 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:19:45.994 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-nbd-2:19.0.0-2165.g63e11116.el9.x86_64 1/1 2024-03-21T06:19:45.995 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:45.995 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-21T06:19:45.995 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd-2:19.0.0-2165.g63e11116.el9.x86_64 2024-03-21T06:19:45.995 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:45.995 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-21T06:19:46.228 DEBUG:teuthology.parallel:result is None 2024-03-21T06:19:46.229 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=63e11116edc7c1585450815ed7030c70ea0ebef9 2024-03-21T06:19:46.395 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T06:19:46.415 INFO:teuthology.orchestra.run.smithi096.stdout:19.0.0-2165.g63e11116.el9 2024-03-21T06:19:46.416 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2165.g63e11116.el9 2024-03-21T06:19:46.416 INFO:teuthology.task.install:The correct ceph version 19.0.0-2165.g63e11116 is installed. 2024-03-21T06:19:46.418 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=63e11116edc7c1585450815ed7030c70ea0ebef9 2024-03-21T06:19:46.575 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T06:19:46.601 INFO:teuthology.orchestra.run.smithi184.stdout:19.0.0-2165.g63e11116.el9 2024-03-21T06:19:46.601 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2165.g63e11116.el9 2024-03-21T06:19:46.601 INFO:teuthology.task.install:The correct ceph version 19.0.0-2165.g63e11116 is installed. 2024-03-21T06:19:46.603 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-21T06:19:46.603 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:19:46.604 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T06:19:46.631 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-21T06:19:46.632 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T06:19:46.674 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-21T06:19:46.675 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:19:46.675 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T06:19:46.701 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T06:19:46.768 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-21T06:19:46.768 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T06:19:46.799 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T06:19:46.870 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-21T06:19:46.870 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:19:46.870 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T06:19:46.897 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T06:19:46.964 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-21T06:19:46.964 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T06:19:46.995 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T06:19:47.066 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-21T06:19:47.066 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:19:47.066 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T06:19:47.093 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T06:19:47.160 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-21T06:19:47.160 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T06:19:47.191 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T06:19:47.260 INFO:teuthology.run_tasks:Running task ceph... 2024-03-21T06:19:47.354 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-21T06:19:47.354 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /var/log/ceph 2024-03-21T06:19:47.357 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /var/log/ceph 2024-03-21T06:19:47.385 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-21T06:19:47.386 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-21T06:19:47.423 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-21T06:19:47.454 INFO:tasks.ceph:Creating extra log directories... 2024-03-21T06:19:47.455 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-21T06:19:47.490 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-21T06:19:47.558 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-21T06:19:47.558 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-21T06:19:47.558 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi/7614674', 'branch': 'ci-yingxin-fix-racing-get-or-create-pg-mapping-2', 'description': 'crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests}', 'email': 'yingxin.cheng@intel.com', 'first_in_suite': False, 'job_id': '7614674', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'ci-yingxin-fix-racing-get-or-create-pg-mapping-2'}, 'ceph': {'conf': {'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '63e11116edc7c1585450815ed7030c70ea0ebef9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '63e11116edc7c1585450815ed7030c70ea0ebef9'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'workunit': {'branch': 'ci-yingxin-fix-racing-get-or-create-pg-mapping-2', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '63e11116edc7c1585450815ed7030c70ea0ebef9'}}, 'owner': 'scheduled_yingxin@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '63e11116edc7c1585450815ed7030c70ea0ebef9', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'ci-yingxin-fix-racing-get-or-create-pg-mapping-2', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_63e11116edc7c1585450815ed7030c70ea0ebef9/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '63e11116edc7c1585450815ed7030c70ea0ebef9', 'targets': {'smithi096.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIUHKbEkR+HUbnaQRi1xHhA4XWyrvOa7J+DYI/mzrXugPbvImI7V4WIeSTLo+3Bm10E3FBR8P1PMH2Py0TAZ+vs=', 'smithi184.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBPpXxYSfuuxjw/v/WXhHe553KpRC/nY5aQTyIFm3OYUyj9OKGREjr66EM1brOjwyTGoe4KJHdb0kzRgfN1wrak='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '63e11116edc7c1585450815ed7030c70ea0ebef9', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['rados/test.sh --crimson', 'rados/test_pool_quota.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-21_05:50:24', 'tube': 'smithi', 'user': 'yingxin', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-21T06:19:47.559 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-21T06:19:47.561 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-21T06:19:47.578 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/run/ceph 2024-03-21T06:19:47.617 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/run/ceph 2024-03-21T06:19:47.645 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:19:47.645 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T06:19:47.696 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T06:19:47.696 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-03-21T06:19:47.750 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T06:19:47.751 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T06:19:47.751 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-21T06:19:47.751 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T06:19:47.751 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T06:19:47.751 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-03-21 06:19:42.955630220 +0000 2024-03-21T06:19:47.751 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-03-21 06:11:36.238230985 +0000 2024-03-21T06:19:47.751 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-03-21 06:11:36.238230985 +0000 2024-03-21T06:19:47.751 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-03-21 06:11:36.238230985 +0000 2024-03-21T06:19:47.751 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T06:19:47.816 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-03-21T06:19:47.817 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-03-21T06:19:47.817 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000205951 s, 2.5 MB/s 2024-03-21T06:19:47.818 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T06:19:47.874 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-03-21T06:19:47.928 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T06:19:47.929 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T06:19:47.929 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-21T06:19:47.929 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T06:19:47.929 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T06:19:47.929 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-03-21 06:19:42.955630220 +0000 2024-03-21T06:19:47.929 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-03-21 06:11:36.599222437 +0000 2024-03-21T06:19:47.929 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-03-21 06:11:36.599222437 +0000 2024-03-21T06:19:47.929 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-03-21 06:11:36.599222437 +0000 2024-03-21T06:19:47.929 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T06:19:47.994 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-03-21T06:19:47.994 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-03-21T06:19:47.995 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000215923 s, 2.4 MB/s 2024-03-21T06:19:47.995 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T06:19:48.052 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-03-21T06:19:48.107 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T06:19:48.107 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T06:19:48.107 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-21T06:19:48.107 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T06:19:48.107 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T06:19:48.107 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-03-21 06:19:42.955630220 +0000 2024-03-21T06:19:48.107 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-03-21 06:11:36.937214433 +0000 2024-03-21T06:19:48.107 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-03-21 06:11:36.937214433 +0000 2024-03-21T06:19:48.107 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-03-21 06:11:36.937214433 +0000 2024-03-21T06:19:48.108 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T06:19:48.172 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-03-21T06:19:48.172 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-03-21T06:19:48.173 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000187557 s, 2.7 MB/s 2024-03-21T06:19:48.174 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T06:19:48.229 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-03-21T06:19:48.284 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T06:19:48.284 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T06:19:48.284 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-21T06:19:48.284 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T06:19:48.284 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T06:19:48.284 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-03-21 06:19:42.956630196 +0000 2024-03-21T06:19:48.284 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-03-21 06:11:37.279206334 +0000 2024-03-21T06:19:48.284 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-03-21 06:11:37.279206334 +0000 2024-03-21T06:19:48.285 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-03-21 06:11:37.279206334 +0000 2024-03-21T06:19:48.285 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T06:19:48.351 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-03-21T06:19:48.351 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-03-21T06:19:48.351 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000214936 s, 2.4 MB/s 2024-03-21T06:19:48.351 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T06:19:48.406 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-03-21T06:19:48.406 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-21T06:19:48.407 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T06:19:48.424 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T06:19:48.424 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2024-03-21T06:19:48.481 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T06:19:48.481 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T06:19:48.481 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-21T06:19:48.481 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T06:19:48.481 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T06:19:48.481 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-03-21 06:15:23.477284724 +0000 2024-03-21T06:19:48.481 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-03-21 06:11:36.248267707 +0000 2024-03-21T06:19:48.482 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-03-21 06:11:36.248267707 +0000 2024-03-21T06:19:48.482 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-03-21 06:11:36.248267707 +0000 2024-03-21T06:19:48.482 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T06:19:48.550 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-03-21T06:19:48.550 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-03-21T06:19:48.550 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000216376 s, 2.4 MB/s 2024-03-21T06:19:48.554 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T06:19:48.612 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2024-03-21T06:19:48.669 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T06:19:48.669 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T06:19:48.669 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-21T06:19:48.669 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T06:19:48.669 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T06:19:48.669 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-03-21 06:15:23.477284724 +0000 2024-03-21T06:19:48.669 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-03-21 06:11:36.583260906 +0000 2024-03-21T06:19:48.669 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-03-21 06:11:36.583260906 +0000 2024-03-21T06:19:48.669 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-03-21 06:11:36.583260906 +0000 2024-03-21T06:19:48.670 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T06:19:48.736 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-03-21T06:19:48.736 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-03-21T06:19:48.736 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000219005 s, 2.3 MB/s 2024-03-21T06:19:48.738 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T06:19:48.795 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2024-03-21T06:19:48.851 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T06:19:48.851 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T06:19:48.851 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-21T06:19:48.851 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T06:19:48.851 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T06:19:48.851 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-03-21 06:15:23.478284704 +0000 2024-03-21T06:19:48.851 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-03-21 06:11:36.928253902 +0000 2024-03-21T06:19:48.851 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-03-21 06:11:36.928253902 +0000 2024-03-21T06:19:48.852 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-03-21 06:11:36.928253902 +0000 2024-03-21T06:19:48.852 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T06:19:48.917 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-03-21T06:19:48.917 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-03-21T06:19:48.917 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000212005 s, 2.4 MB/s 2024-03-21T06:19:48.918 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T06:19:48.977 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2024-03-21T06:19:49.033 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T06:19:49.033 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T06:19:49.033 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-21T06:19:49.033 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T06:19:49.034 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T06:19:49.034 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-03-21 06:15:23.478284704 +0000 2024-03-21T06:19:49.034 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-03-21 06:11:37.285246654 +0000 2024-03-21T06:19:49.034 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-03-21 06:11:37.285246654 +0000 2024-03-21T06:19:49.034 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-03-21 06:11:37.285246654 +0000 2024-03-21T06:19:49.035 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T06:19:49.099 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-03-21T06:19:49.100 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-03-21T06:19:49.100 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000244571 s, 2.1 MB/s 2024-03-21T06:19:49.102 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T06:19:49.158 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-03-21T06:19:49.159 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-03-21T06:19:49.159 INFO:tasks.ceph:Generating config... 2024-03-21T06:19:49.161 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-03-21T06:19:49.161 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-03-21T06:19:49.161 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-03-21T06:19:49.161 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-03-21T06:19:49.162 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-03-21T06:19:49.162 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-21T06:19:49.162 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-21T06:19:49.162 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-21T06:19:49.162 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-03-21T06:19:49.163 INFO:tasks.ceph:[mon] osd pool default crimson = True 2024-03-21T06:19:49.163 INFO:tasks.ceph:[osd] debug monc = 20 2024-03-21T06:19:49.163 INFO:tasks.ceph:[osd] crimson alien thread cpu cores = 6-7 2024-03-21T06:19:49.163 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2024-03-21T06:19:49.163 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-21T06:19:49.163 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-21T06:19:49.163 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-03-21T06:19:49.164 INFO:tasks.ceph:[osd] osd class default list = * 2024-03-21T06:19:49.164 INFO:tasks.ceph:[osd] osd class load list = * 2024-03-21T06:19:49.164 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-03-21T06:19:49.164 INFO:tasks.ceph:[client] debug ms = 1 2024-03-21T06:19:49.164 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2024-03-21T06:19:49.164 INFO:tasks.ceph:[global] ms client mode = crc 2024-03-21T06:19:49.165 INFO:tasks.ceph:[global] ms cluster mode = crc 2024-03-21T06:19:49.165 INFO:tasks.ceph:[global] ms mon client mode = crc 2024-03-21T06:19:49.165 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2024-03-21T06:19:49.165 INFO:tasks.ceph:[global] ms mon service mode = crc 2024-03-21T06:19:49.165 INFO:tasks.ceph:[global] ms service mode = crc 2024-03-21T06:19:49.165 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-21T06:19:49.166 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-21T06:19:49.166 INFO:tasks.ceph:[osd.0] crimson seastar cpu cores = 0-2 2024-03-21T06:19:49.166 INFO:tasks.ceph:[osd.1] crimson seastar cpu cores = 3-5 2024-03-21T06:19:49.166 INFO:tasks.ceph:[osd.2] crimson seastar cpu cores = 0-2 2024-03-21T06:19:49.166 INFO:tasks.ceph:[osd.3] crimson seastar cpu cores = 3-5 2024-03-21T06:19:49.167 INFO:tasks.ceph:Setting up mon.a... 2024-03-21T06:19:49.167 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-21T06:19:49.235 INFO:teuthology.orchestra.run.smithi096.stdout:creating /etc/ceph/ceph.keyring 2024-03-21T06:19:49.238 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-21T06:19:49.292 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-21T06:19:49.319 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.96')] 2024-03-21T06:19:49.319 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.96', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1}, 'mon.a': {}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}} 2024-03-21T06:19:49.320 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:19:49.320 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-21T06:19:49.374 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.96 --print /home/ubuntu/cephtest/ceph.monmap 2024-03-21T06:19:49.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:19:49.460+0000 7f5291e6ee80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:19:49.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:19:49.460+0000 7f5291e6ee80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:19:49.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:19:49.460+0000 7f5291e6ee80 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:19:49.461 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-21T06:19:49.462 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool: generated fsid e880df84-2d7c-4386-b5e9-f3879b9ab9d5 2024-03-21T06:19:49.462 INFO:teuthology.orchestra.run.smithi096.stdout:setting min_mon_release = quincy 2024-03-21T06:19:49.462 INFO:teuthology.orchestra.run.smithi096.stdout:epoch 0 2024-03-21T06:19:49.462 INFO:teuthology.orchestra.run.smithi096.stdout:fsid e880df84-2d7c-4386-b5e9-f3879b9ab9d5 2024-03-21T06:19:49.462 INFO:teuthology.orchestra.run.smithi096.stdout:last_changed 2024-03-21T06:19:49.462086+0000 2024-03-21T06:19:49.462 INFO:teuthology.orchestra.run.smithi096.stdout:created 2024-03-21T06:19:49.462086+0000 2024-03-21T06:19:49.462 INFO:teuthology.orchestra.run.smithi096.stdout:min_mon_release 17 (quincy) 2024-03-21T06:19:49.462 INFO:teuthology.orchestra.run.smithi096.stdout:election_strategy: 1 2024-03-21T06:19:49.462 INFO:teuthology.orchestra.run.smithi096.stdout:0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-03-21T06:19:49.462 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-03-21T06:19:49.463 DEBUG:teuthology.orchestra.run.smithi096:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-21T06:19:49.478 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e880df84-2d7c-4386-b5e9-f3879b9ab9d5... 2024-03-21T06:19:49.478 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-21T06:19:49.520 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-21T06:19:49.568 INFO:teuthology.orchestra.run.smithi096.stdout:[global] 2024-03-21T06:19:49.568 INFO:teuthology.orchestra.run.smithi096.stdout: chdir = "" 2024-03-21T06:19:49.568 INFO:teuthology.orchestra.run.smithi096.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-21T06:19:49.568 INFO:teuthology.orchestra.run.smithi096.stdout: auth supported = cephx 2024-03-21T06:19:49.568 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.568 INFO:teuthology.orchestra.run.smithi096.stdout: filestore xattr use omap = true 2024-03-21T06:19:49.568 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.568 INFO:teuthology.orchestra.run.smithi096.stdout: mon clock drift allowed = 1.000 2024-03-21T06:19:49.568 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.568 INFO:teuthology.orchestra.run.smithi096.stdout: osd crush chooseleaf type = 0 2024-03-21T06:19:49.568 INFO:teuthology.orchestra.run.smithi096.stdout: auth debug = true 2024-03-21T06:19:49.569 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.569 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on old message = true 2024-03-21T06:19:49.569 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on bug = true 2024-03-21T06:19:49.569 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.569 INFO:teuthology.orchestra.run.smithi096.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-21T06:19:49.569 INFO:teuthology.orchestra.run.smithi096.stdout: mon pg warn max object skew = 0 2024-03-21T06:19:49.569 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.569 INFO:teuthology.orchestra.run.smithi096.stdout: # disable pg_autoscaler by default for new pools 2024-03-21T06:19:49.569 INFO:teuthology.orchestra.run.smithi096.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-21T06:19:49.569 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.569 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default size = 2 2024-03-21T06:19:49.569 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.569 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow primary affinity = true 2024-03-21T06:19:49.569 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow pg remap = true 2024-03-21T06:19:49.569 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on legacy crush tunables = false 2024-03-21T06:19:49.570 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on crush straw calc version zero = false 2024-03-21T06:19:49.570 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on no sortbitwise = false 2024-03-21T06:19:49.570 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on osd down out interval zero = false 2024-03-21T06:19:49.570 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on too few osds = false 2024-03-21T06:19:49.570 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-21T06:19:49.570 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-21T06:19:49.570 INFO:teuthology.orchestra.run.smithi096.stdout: mon_allow_pool_size_one = true 2024-03-21T06:19:49.570 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.570 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-21T06:19:49.570 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.570 INFO:teuthology.orchestra.run.smithi096.stdout: osd default data pool replay window = 5 2024-03-21T06:19:49.570 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.570 INFO:teuthology.orchestra.run.smithi096.stdout: mon allow pool delete = true 2024-03-21T06:19:49.570 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.570 INFO:teuthology.orchestra.run.smithi096.stdout: mon cluster log file level = debug 2024-03-21T06:19:49.571 INFO:teuthology.orchestra.run.smithi096.stdout: debug asserts on shutdown = true 2024-03-21T06:19:49.571 INFO:teuthology.orchestra.run.smithi096.stdout: mon health detail to clog = false 2024-03-21T06:19:49.571 INFO:teuthology.orchestra.run.smithi096.stdout: mon host = 172.21.15.96 2024-03-21T06:19:49.571 INFO:teuthology.orchestra.run.smithi096.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-03-21T06:19:49.571 INFO:teuthology.orchestra.run.smithi096.stdout: ms client mode = crc 2024-03-21T06:19:49.571 INFO:teuthology.orchestra.run.smithi096.stdout: ms cluster mode = crc 2024-03-21T06:19:49.571 INFO:teuthology.orchestra.run.smithi096.stdout: ms mon client mode = crc 2024-03-21T06:19:49.571 INFO:teuthology.orchestra.run.smithi096.stdout: ms mon cluster mode = crc 2024-03-21T06:19:49.571 INFO:teuthology.orchestra.run.smithi096.stdout: ms mon service mode = crc 2024-03-21T06:19:49.571 INFO:teuthology.orchestra.run.smithi096.stdout: ms service mode = crc 2024-03-21T06:19:49.571 INFO:teuthology.orchestra.run.smithi096.stdout: fsid = e880df84-2d7c-4386-b5e9-f3879b9ab9d5 2024-03-21T06:19:49.571 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.571 INFO:teuthology.orchestra.run.smithi096.stdout:[osd] 2024-03-21T06:19:49.571 INFO:teuthology.orchestra.run.smithi096.stdout: osd journal size = 100 2024-03-21T06:19:49.571 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.571 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub load threshold = 5.0 2024-03-21T06:19:49.572 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub max interval = 600 2024-03-21T06:19:49.572 INFO:teuthology.orchestra.run.smithi096.stdout: osd mclock profile = high_recovery_ops 2024-03-21T06:19:49.572 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.572 INFO:teuthology.orchestra.run.smithi096.stdout: osd recover clone overlap = true 2024-03-21T06:19:49.572 INFO:teuthology.orchestra.run.smithi096.stdout: osd recovery max chunk = 1048576 2024-03-21T06:19:49.572 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.572 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug shutdown = true 2024-03-21T06:19:49.572 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug op order = true 2024-03-21T06:19:49.572 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug verify stray on activate = true 2024-03-21T06:19:49.572 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug trim objects = true 2024-03-21T06:19:49.572 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.572 INFO:teuthology.orchestra.run.smithi096.stdout: osd open classes on start = true 2024-03-21T06:19:49.572 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug pg log writeout = true 2024-03-21T06:19:49.572 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.572 INFO:teuthology.orchestra.run.smithi096.stdout: osd deep scrub update digest min age = 30 2024-03-21T06:19:49.572 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.573 INFO:teuthology.orchestra.run.smithi096.stdout: osd map max advance = 10 2024-03-21T06:19:49.573 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.573 INFO:teuthology.orchestra.run.smithi096.stdout: journal zero on create = true 2024-03-21T06:19:49.573 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.573 INFO:teuthology.orchestra.run.smithi096.stdout: filestore ondisk finisher threads = 3 2024-03-21T06:19:49.573 INFO:teuthology.orchestra.run.smithi096.stdout: filestore apply finisher threads = 3 2024-03-21T06:19:49.573 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.573 INFO:teuthology.orchestra.run.smithi096.stdout: bdev debug aio = true 2024-03-21T06:19:49.573 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug misdirected ops = true 2024-03-21T06:19:49.573 INFO:teuthology.orchestra.run.smithi096.stdout: debug monc = 20 2024-03-21T06:19:49.573 INFO:teuthology.orchestra.run.smithi096.stdout: crimson alien thread cpu cores = 6-7 2024-03-21T06:19:49.573 INFO:teuthology.orchestra.run.smithi096.stdout: crimson osd obc lru size = 10 2024-03-21T06:19:49.573 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-03-21T06:19:49.573 INFO:teuthology.orchestra.run.smithi096.stdout: debug osd = 20 2024-03-21T06:19:49.573 INFO:teuthology.orchestra.run.smithi096.stdout: osd blocked scrub grace period = 3600 2024-03-21T06:19:49.573 INFO:teuthology.orchestra.run.smithi096.stdout: osd class default list = * 2024-03-21T06:19:49.574 INFO:teuthology.orchestra.run.smithi096.stdout: osd class load list = * 2024-03-21T06:19:49.574 INFO:teuthology.orchestra.run.smithi096.stdout: osd shutdown pgref assert = True 2024-03-21T06:19:49.574 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.574 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr] 2024-03-21T06:19:49.574 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-03-21T06:19:49.574 INFO:teuthology.orchestra.run.smithi096.stdout: debug mgr = 20 2024-03-21T06:19:49.574 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2024-03-21T06:19:49.574 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2024-03-21T06:19:49.574 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2024-03-21T06:19:49.574 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2024-03-21T06:19:49.574 INFO:teuthology.orchestra.run.smithi096.stdout: mgr/telemetry/nag = false 2024-03-21T06:19:49.574 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.574 INFO:teuthology.orchestra.run.smithi096.stdout:[mon] 2024-03-21T06:19:49.574 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-03-21T06:19:49.574 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2024-03-21T06:19:49.575 INFO:teuthology.orchestra.run.smithi096.stdout: debug paxos = 20 2024-03-21T06:19:49.575 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2024-03-21T06:19:49.575 INFO:teuthology.orchestra.run.smithi096.stdout: mon data avail warn = 5 2024-03-21T06:19:49.575 INFO:teuthology.orchestra.run.smithi096.stdout: mon mgr mkfs grace = 240 2024-03-21T06:19:49.575 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2024-03-21T06:19:49.575 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd reporter subtree level = osd 2024-03-21T06:19:49.575 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd prime pg temp = true 2024-03-21T06:19:49.575 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2024-03-21T06:19:49.575 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.575 INFO:teuthology.orchestra.run.smithi096.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-21T06:19:49.575 INFO:teuthology.orchestra.run.smithi096.stdout: auth mon ticket ttl = 660 # 11m 2024-03-21T06:19:49.575 INFO:teuthology.orchestra.run.smithi096.stdout: auth service ticket ttl = 240 # 4m 2024-03-21T06:19:49.575 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.575 INFO:teuthology.orchestra.run.smithi096.stdout: # don't complain about insecure global_id in the test suite 2024-03-21T06:19:49.575 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-21T06:19:49.576 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-21T06:19:49.576 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.576 INFO:teuthology.orchestra.run.smithi096.stdout: # 1m isn't quite enough 2024-03-21T06:19:49.576 INFO:teuthology.orchestra.run.smithi096.stdout: mon_down_mkfs_grace = 2m 2024-03-21T06:19:49.577 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.577 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_filestore_osds = false 2024-03-21T06:19:49.577 INFO:teuthology.orchestra.run.smithi096.stdout: mon min osdmap epochs = 50 2024-03-21T06:19:49.577 INFO:teuthology.orchestra.run.smithi096.stdout: mon osdmap full prune interval = 2 2024-03-21T06:19:49.577 INFO:teuthology.orchestra.run.smithi096.stdout: mon osdmap full prune min = 15 2024-03-21T06:19:49.577 INFO:teuthology.orchestra.run.smithi096.stdout: mon osdmap full prune txsize = 2 2024-03-21T06:19:49.578 INFO:teuthology.orchestra.run.smithi096.stdout: paxos service trim min = 10 2024-03-21T06:19:49.578 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on pool no app = False 2024-03-21T06:19:49.578 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default crimson = True 2024-03-21T06:19:49.578 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:19:49.578 INFO:teuthology.orchestra.run.smithi096.stdout:[client] 2024-03-21T06:19:49.578 INFO:teuthology.orchestra.run.smithi096.stdout: rgw cache enabled = true 2024-03-21T06:19:49.578 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable ops log = true 2024-03-21T06:19:49.578 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable usage log = true 2024-03-21T06:19:49.578 INFO:teuthology.orchestra.run.smithi096.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-21T06:19:49.578 INFO:teuthology.orchestra.run.smithi096.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-21T06:19:49.578 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-03-21T06:19:49.578 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.a] 2024-03-21T06:19:49.578 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.0] 2024-03-21T06:19:49.578 INFO:teuthology.orchestra.run.smithi096.stdout: crimson seastar cpu cores = 0-2 2024-03-21T06:19:49.578 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.1] 2024-03-21T06:19:49.579 INFO:teuthology.orchestra.run.smithi096.stdout: crimson seastar cpu cores = 3-5 2024-03-21T06:19:49.579 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.2] 2024-03-21T06:19:49.579 INFO:teuthology.orchestra.run.smithi096.stdout: crimson seastar cpu cores = 0-2 2024-03-21T06:19:49.579 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.3] 2024-03-21T06:19:49.579 INFO:teuthology.orchestra.run.smithi096.stdout: crimson seastar cpu cores = 3-5 2024-03-21T06:19:49.579 INFO:teuthology.orchestra.run.smithi184.stdout:[global] 2024-03-21T06:19:49.579 INFO:teuthology.orchestra.run.smithi184.stdout: chdir = "" 2024-03-21T06:19:49.579 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-21T06:19:49.579 INFO:teuthology.orchestra.run.smithi184.stdout: auth supported = cephx 2024-03-21T06:19:49.579 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.580 INFO:teuthology.orchestra.run.smithi184.stdout: filestore xattr use omap = true 2024-03-21T06:19:49.580 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.580 INFO:teuthology.orchestra.run.smithi184.stdout: mon clock drift allowed = 1.000 2024-03-21T06:19:49.580 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.580 INFO:teuthology.orchestra.run.smithi184.stdout: osd crush chooseleaf type = 0 2024-03-21T06:19:49.580 INFO:teuthology.orchestra.run.smithi184.stdout: auth debug = true 2024-03-21T06:19:49.580 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.580 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on old message = true 2024-03-21T06:19:49.580 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on bug = true 2024-03-21T06:19:49.580 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.580 INFO:teuthology.orchestra.run.smithi184.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-21T06:19:49.580 INFO:teuthology.orchestra.run.smithi184.stdout: mon pg warn max object skew = 0 2024-03-21T06:19:49.580 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.580 INFO:teuthology.orchestra.run.smithi184.stdout: # disable pg_autoscaler by default for new pools 2024-03-21T06:19:49.580 INFO:teuthology.orchestra.run.smithi184.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-21T06:19:49.580 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.581 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default size = 2 2024-03-21T06:19:49.581 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.581 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow primary affinity = true 2024-03-21T06:19:49.581 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow pg remap = true 2024-03-21T06:19:49.581 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on legacy crush tunables = false 2024-03-21T06:19:49.581 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on crush straw calc version zero = false 2024-03-21T06:19:49.581 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on no sortbitwise = false 2024-03-21T06:19:49.581 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on osd down out interval zero = false 2024-03-21T06:19:49.581 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on too few osds = false 2024-03-21T06:19:49.581 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-21T06:19:49.581 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-21T06:19:49.581 INFO:teuthology.orchestra.run.smithi184.stdout: mon_allow_pool_size_one = true 2024-03-21T06:19:49.581 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.581 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-21T06:19:49.581 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.581 INFO:teuthology.orchestra.run.smithi184.stdout: osd default data pool replay window = 5 2024-03-21T06:19:49.581 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.582 INFO:teuthology.orchestra.run.smithi184.stdout: mon allow pool delete = true 2024-03-21T06:19:49.582 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.582 INFO:teuthology.orchestra.run.smithi184.stdout: mon cluster log file level = debug 2024-03-21T06:19:49.582 INFO:teuthology.orchestra.run.smithi184.stdout: debug asserts on shutdown = true 2024-03-21T06:19:49.582 INFO:teuthology.orchestra.run.smithi184.stdout: mon health detail to clog = false 2024-03-21T06:19:49.582 INFO:teuthology.orchestra.run.smithi184.stdout: mon host = 172.21.15.96 2024-03-21T06:19:49.582 INFO:teuthology.orchestra.run.smithi184.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-03-21T06:19:49.582 INFO:teuthology.orchestra.run.smithi184.stdout: ms client mode = crc 2024-03-21T06:19:49.582 INFO:teuthology.orchestra.run.smithi184.stdout: ms cluster mode = crc 2024-03-21T06:19:49.582 INFO:teuthology.orchestra.run.smithi184.stdout: ms mon client mode = crc 2024-03-21T06:19:49.582 INFO:teuthology.orchestra.run.smithi184.stdout: ms mon cluster mode = crc 2024-03-21T06:19:49.582 INFO:teuthology.orchestra.run.smithi184.stdout: ms mon service mode = crc 2024-03-21T06:19:49.582 INFO:teuthology.orchestra.run.smithi184.stdout: ms service mode = crc 2024-03-21T06:19:49.582 INFO:teuthology.orchestra.run.smithi184.stdout: fsid = e880df84-2d7c-4386-b5e9-f3879b9ab9d5 2024-03-21T06:19:49.582 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.582 INFO:teuthology.orchestra.run.smithi184.stdout:[osd] 2024-03-21T06:19:49.583 INFO:teuthology.orchestra.run.smithi184.stdout: osd journal size = 100 2024-03-21T06:19:49.583 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.583 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub load threshold = 5.0 2024-03-21T06:19:49.583 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub max interval = 600 2024-03-21T06:19:49.583 INFO:teuthology.orchestra.run.smithi184.stdout: osd mclock profile = high_recovery_ops 2024-03-21T06:19:49.583 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.583 INFO:teuthology.orchestra.run.smithi184.stdout: osd recover clone overlap = true 2024-03-21T06:19:49.583 INFO:teuthology.orchestra.run.smithi184.stdout: osd recovery max chunk = 1048576 2024-03-21T06:19:49.583 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.583 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug shutdown = true 2024-03-21T06:19:49.583 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug op order = true 2024-03-21T06:19:49.583 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug verify stray on activate = true 2024-03-21T06:19:49.583 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug trim objects = true 2024-03-21T06:19:49.583 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.583 INFO:teuthology.orchestra.run.smithi184.stdout: osd open classes on start = true 2024-03-21T06:19:49.583 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug pg log writeout = true 2024-03-21T06:19:49.583 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: osd deep scrub update digest min age = 30 2024-03-21T06:19:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: osd map max advance = 10 2024-03-21T06:19:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: journal zero on create = true 2024-03-21T06:19:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: filestore ondisk finisher threads = 3 2024-03-21T06:19:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: filestore apply finisher threads = 3 2024-03-21T06:19:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: bdev debug aio = true 2024-03-21T06:19:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug misdirected ops = true 2024-03-21T06:19:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: debug monc = 20 2024-03-21T06:19:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: crimson alien thread cpu cores = 6-7 2024-03-21T06:19:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: crimson osd obc lru size = 10 2024-03-21T06:19:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-03-21T06:19:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: debug osd = 20 2024-03-21T06:19:49.584 INFO:teuthology.orchestra.run.smithi184.stdout: osd blocked scrub grace period = 3600 2024-03-21T06:19:49.585 INFO:teuthology.orchestra.run.smithi184.stdout: osd class default list = * 2024-03-21T06:19:49.585 INFO:teuthology.orchestra.run.smithi184.stdout: osd class load list = * 2024-03-21T06:19:49.585 INFO:teuthology.orchestra.run.smithi184.stdout: osd shutdown pgref assert = True 2024-03-21T06:19:49.585 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.585 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr] 2024-03-21T06:19:49.585 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-03-21T06:19:49.585 INFO:teuthology.orchestra.run.smithi184.stdout: debug mgr = 20 2024-03-21T06:19:49.585 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2024-03-21T06:19:49.585 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2024-03-21T06:19:49.585 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2024-03-21T06:19:49.585 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2024-03-21T06:19:49.585 INFO:teuthology.orchestra.run.smithi184.stdout: mgr/telemetry/nag = false 2024-03-21T06:19:49.585 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.585 INFO:teuthology.orchestra.run.smithi184.stdout:[mon] 2024-03-21T06:19:49.585 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-03-21T06:19:49.585 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2024-03-21T06:19:49.586 INFO:teuthology.orchestra.run.smithi184.stdout: debug paxos = 20 2024-03-21T06:19:49.586 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2024-03-21T06:19:49.586 INFO:teuthology.orchestra.run.smithi184.stdout: mon data avail warn = 5 2024-03-21T06:19:49.586 INFO:teuthology.orchestra.run.smithi184.stdout: mon mgr mkfs grace = 240 2024-03-21T06:19:49.586 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2024-03-21T06:19:49.586 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd reporter subtree level = osd 2024-03-21T06:19:49.586 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd prime pg temp = true 2024-03-21T06:19:49.586 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2024-03-21T06:19:49.586 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.586 INFO:teuthology.orchestra.run.smithi184.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-21T06:19:49.586 INFO:teuthology.orchestra.run.smithi184.stdout: auth mon ticket ttl = 660 # 11m 2024-03-21T06:19:49.586 INFO:teuthology.orchestra.run.smithi184.stdout: auth service ticket ttl = 240 # 4m 2024-03-21T06:19:49.586 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.586 INFO:teuthology.orchestra.run.smithi184.stdout: # don't complain about insecure global_id in the test suite 2024-03-21T06:19:49.586 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-21T06:19:49.587 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-21T06:19:49.587 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.587 INFO:teuthology.orchestra.run.smithi184.stdout: # 1m isn't quite enough 2024-03-21T06:19:49.587 INFO:teuthology.orchestra.run.smithi184.stdout: mon_down_mkfs_grace = 2m 2024-03-21T06:19:49.587 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.587 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_filestore_osds = false 2024-03-21T06:19:49.587 INFO:teuthology.orchestra.run.smithi184.stdout: mon min osdmap epochs = 50 2024-03-21T06:19:49.587 INFO:teuthology.orchestra.run.smithi184.stdout: mon osdmap full prune interval = 2 2024-03-21T06:19:49.587 INFO:teuthology.orchestra.run.smithi184.stdout: mon osdmap full prune min = 15 2024-03-21T06:19:49.587 INFO:teuthology.orchestra.run.smithi184.stdout: mon osdmap full prune txsize = 2 2024-03-21T06:19:49.587 INFO:teuthology.orchestra.run.smithi184.stdout: paxos service trim min = 10 2024-03-21T06:19:49.587 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on pool no app = False 2024-03-21T06:19:49.587 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default crimson = True 2024-03-21T06:19:49.587 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-03-21T06:19:49.587 INFO:teuthology.orchestra.run.smithi184.stdout:[client] 2024-03-21T06:19:49.587 INFO:teuthology.orchestra.run.smithi184.stdout: rgw cache enabled = true 2024-03-21T06:19:49.587 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable ops log = true 2024-03-21T06:19:49.588 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable usage log = true 2024-03-21T06:19:49.588 INFO:teuthology.orchestra.run.smithi184.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-21T06:19:49.588 INFO:teuthology.orchestra.run.smithi184.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-21T06:19:49.588 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2024-03-21T06:19:49.588 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.a] 2024-03-21T06:19:49.588 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.0] 2024-03-21T06:19:49.588 INFO:teuthology.orchestra.run.smithi184.stdout: crimson seastar cpu cores = 0-2 2024-03-21T06:19:49.588 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.1] 2024-03-21T06:19:49.588 INFO:teuthology.orchestra.run.smithi184.stdout: crimson seastar cpu cores = 3-5 2024-03-21T06:19:49.588 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.2] 2024-03-21T06:19:49.588 INFO:teuthology.orchestra.run.smithi184.stdout: crimson seastar cpu cores = 0-2 2024-03-21T06:19:49.588 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.3] 2024-03-21T06:19:49.588 INFO:teuthology.orchestra.run.smithi184.stdout: crimson seastar cpu cores = 3-5 2024-03-21T06:19:49.589 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-21T06:19:49.590 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-21T06:19:49.651 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-21T06:19:49.651 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:19:49.651 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-21T06:19:49.666 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:19:49.666 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-21T06:19:49.720 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi096.front.sepia.ceph.com 2024-03-21T06:19:49.720 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:19:49.720 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-21T06:19:49.720 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-21T06:19:49.799 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:19:49.799 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-21T06:19:49.854 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi184.front.sepia.ceph.com 2024-03-21T06:19:49.854 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-21T06:19:49.854 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-21T06:19:49.854 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-21T06:19:49.894 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-21T06:19:49.894 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-21T06:19:49.949 INFO:tasks.ceph:Setting up mon nodes... 2024-03-21T06:19:49.949 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-21T06:19:49.949 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-21T06:19:50.082 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-21T06:19:50.085 INFO:tasks.ceph:Setting up mds nodes... 2024-03-21T06:19:50.086 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-21T06:19:50.086 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-21T06:19:50.134 INFO:teuthology.orchestra.run.smithi096.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-21T06:19:50.150 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-21T06:19:50.198 INFO:teuthology.orchestra.run.smithi184.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-21T06:19:50.214 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-21T06:19:50.214 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-03-21T06:19:50.214 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-21T06:19:50.241 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-03-21T06:19:50.241 INFO:tasks.ceph:role: osd.0 2024-03-21T06:19:50.241 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com 2024-03-21T06:19:50.241 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-21T06:19:50.315 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-21T06:19:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-21T06:19:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-21T06:19:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-21T06:19:50.315 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-21T06:19:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-03-21T06:19:50.315 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-21T06:19:50.315 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-21T06:19:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-21T06:19:50.315 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-21T06:19:50.833 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-03-21T06:19:50.835 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-03-21T06:19:50.835 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-21T06:19:50.978 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-21T06:19:51.014 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-21T06:19:51.083 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-03-21T06:19:51.083 INFO:tasks.ceph:role: osd.1 2024-03-21T06:19:51.083 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi096.front.sepia.ceph.com 2024-03-21T06:19:51.083 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-21T06:19:51.158 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-21T06:19:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-21T06:19:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-21T06:19:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-21T06:19:51.159 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-21T06:19:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-03-21T06:19:51.159 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-21T06:19:51.159 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-21T06:19:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-21T06:19:51.159 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-21T06:19:51.612 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-03-21T06:19:51.614 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-03-21T06:19:51.614 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-21T06:19:51.703 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-21T06:19:51.734 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-21T06:19:52.266 INFO:teuthology.orchestra.run.smithi096.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-21T06:19:52.266 INFO:teuthology.orchestra.run.smithi096.stderr:INFO 2024-03-21 06:19:52,267 seastar - Reactor backend: linux-aio 2024-03-21T06:19:52.288 INFO:teuthology.orchestra.run.smithi096.stderr:==28662==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-21T06:19:52.331 INFO:teuthology.orchestra.run.smithi096.stderr:INFO 2024-03-21 06:19:52,332 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-03-21T06:19:52.331 INFO:teuthology.orchestra.run.smithi096.stderr:INFO 2024-03-21 06:19:52,332 [shard 0:main] osd - get_early_config: set --cpuset 0-2 2024-03-21T06:19:52.332 INFO:teuthology.orchestra.run.smithi096.stderr:DEBUG 2024-03-21 06:19:52,333 [shard 0:main] seastar - reactor::drain 2024-03-21T06:19:52.816 INFO:teuthology.orchestra.run.smithi096.stderr:----------------------------------------------------- 2024-03-21T06:19:52.816 INFO:teuthology.orchestra.run.smithi096.stderr:Suppressions used: 2024-03-21T06:19:52.816 INFO:teuthology.orchestra.run.smithi096.stderr: count bytes template 2024-03-21T06:19:52.816 INFO:teuthology.orchestra.run.smithi096.stderr: 1 8 MallocExtension::Register 2024-03-21T06:19:52.816 INFO:teuthology.orchestra.run.smithi096.stderr:----------------------------------------------------- 2024-03-21T06:19:52.816 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-03-21T06:19:52.870 INFO:teuthology.orchestra.run.smithi096.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-21T06:19:52.871 INFO:teuthology.orchestra.run.smithi096.stderr:INFO 2024-03-21 06:19:52,872 seastar - Reactor backend: linux-aio 2024-03-21T06:19:52.899 INFO:teuthology.orchestra.run.smithi096.stderr:==28660==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-21T06:19:55.968 INFO:teuthology.orchestra.run.smithi096.stderr:----------------------------------------------------- 2024-03-21T06:19:55.968 INFO:teuthology.orchestra.run.smithi096.stderr:Suppressions used: 2024-03-21T06:19:55.968 INFO:teuthology.orchestra.run.smithi096.stderr: count bytes template 2024-03-21T06:19:55.968 INFO:teuthology.orchestra.run.smithi096.stderr: 1 8 MallocExtension::Register 2024-03-21T06:19:55.969 INFO:teuthology.orchestra.run.smithi096.stderr:----------------------------------------------------- 2024-03-21T06:19:55.969 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-03-21T06:19:55.969 INFO:teuthology.orchestra.run.smithi096.stdout:created object store /var/lib/ceph/osd/ceph-0 for osd.0 fsid e880df84-2d7c-4386-b5e9-f3879b9ab9d5 2024-03-21T06:19:56.010 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-21T06:19:56.036 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-21T06:19:56.558 INFO:teuthology.orchestra.run.smithi096.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-21T06:19:56.559 INFO:teuthology.orchestra.run.smithi096.stderr:INFO 2024-03-21 06:19:56,559 seastar - Reactor backend: linux-aio 2024-03-21T06:19:56.582 INFO:teuthology.orchestra.run.smithi096.stderr:==29470==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-21T06:19:56.624 INFO:teuthology.orchestra.run.smithi096.stderr:INFO 2024-03-21 06:19:56,625 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-03-21T06:19:56.624 INFO:teuthology.orchestra.run.smithi096.stderr:INFO 2024-03-21 06:19:56,625 [shard 0:main] osd - get_early_config: set --cpuset 3-5 2024-03-21T06:19:56.625 INFO:teuthology.orchestra.run.smithi096.stderr:DEBUG 2024-03-21 06:19:56,625 [shard 0:main] seastar - reactor::drain 2024-03-21T06:19:57.100 INFO:teuthology.orchestra.run.smithi096.stderr:----------------------------------------------------- 2024-03-21T06:19:57.100 INFO:teuthology.orchestra.run.smithi096.stderr:Suppressions used: 2024-03-21T06:19:57.100 INFO:teuthology.orchestra.run.smithi096.stderr: count bytes template 2024-03-21T06:19:57.100 INFO:teuthology.orchestra.run.smithi096.stderr: 1 8 MallocExtension::Register 2024-03-21T06:19:57.100 INFO:teuthology.orchestra.run.smithi096.stderr:----------------------------------------------------- 2024-03-21T06:19:57.100 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-03-21T06:19:57.153 INFO:teuthology.orchestra.run.smithi096.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-21T06:19:57.154 INFO:teuthology.orchestra.run.smithi096.stderr:INFO 2024-03-21 06:19:57,155 seastar - Reactor backend: linux-aio 2024-03-21T06:19:57.182 INFO:teuthology.orchestra.run.smithi096.stderr:==29468==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-21T06:20:00.326 INFO:teuthology.orchestra.run.smithi096.stderr:----------------------------------------------------- 2024-03-21T06:20:00.327 INFO:teuthology.orchestra.run.smithi096.stderr:Suppressions used: 2024-03-21T06:20:00.327 INFO:teuthology.orchestra.run.smithi096.stderr: count bytes template 2024-03-21T06:20:00.327 INFO:teuthology.orchestra.run.smithi096.stderr: 1 8 MallocExtension::Register 2024-03-21T06:20:00.327 INFO:teuthology.orchestra.run.smithi096.stderr:----------------------------------------------------- 2024-03-21T06:20:00.327 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-03-21T06:20:00.327 INFO:teuthology.orchestra.run.smithi096.stdout:created object store /var/lib/ceph/osd/ceph-1 for osd.1 fsid e880df84-2d7c-4386-b5e9-f3879b9ab9d5 2024-03-21T06:20:00.368 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-21T06:20:00.395 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-21T06:20:00.427 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-03-21T06:20:00.427 INFO:tasks.ceph:role: osd.2 2024-03-21T06:20:00.427 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com 2024-03-21T06:20:00.428 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-21T06:20:00.504 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-21T06:20:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-21T06:20:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-21T06:20:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-21T06:20:00.504 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-21T06:20:00.504 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-03-21T06:20:00.504 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-21T06:20:00.505 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-21T06:20:00.505 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-21T06:20:00.505 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-21T06:20:01.017 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-03-21T06:20:01.019 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-03-21T06:20:01.019 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-03-21T06:20:01.148 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-21T06:20:01.185 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-21T06:20:01.254 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-03-21T06:20:01.257 INFO:tasks.ceph:role: osd.3 2024-03-21T06:20:01.257 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com 2024-03-21T06:20:01.258 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-21T06:20:01.329 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-21T06:20:01.329 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-21T06:20:01.329 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-21T06:20:01.329 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-21T06:20:01.330 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-21T06:20:01.330 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2024-03-21T06:20:01.330 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-21T06:20:01.330 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-21T06:20:01.330 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-21T06:20:01.330 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-21T06:20:01.783 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2024-03-21T06:20:01.785 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2024-03-21T06:20:01.785 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2024-03-21T06:20:01.881 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-21T06:20:01.915 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-21T06:20:02.507 INFO:teuthology.orchestra.run.smithi184.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-21T06:20:02.508 INFO:teuthology.orchestra.run.smithi184.stderr:INFO 2024-03-21 06:20:02,508 seastar - Reactor backend: linux-aio 2024-03-21T06:20:02.529 INFO:teuthology.orchestra.run.smithi184.stderr:==28495==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-21T06:20:02.572 INFO:teuthology.orchestra.run.smithi184.stderr:INFO 2024-03-21 06:20:02,573 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-03-21T06:20:02.572 INFO:teuthology.orchestra.run.smithi184.stderr:INFO 2024-03-21 06:20:02,573 [shard 0:main] osd - get_early_config: set --cpuset 0-2 2024-03-21T06:20:02.573 INFO:teuthology.orchestra.run.smithi184.stderr:DEBUG 2024-03-21 06:20:02,573 [shard 0:main] seastar - reactor::drain 2024-03-21T06:20:03.115 INFO:teuthology.orchestra.run.smithi184.stderr:----------------------------------------------------- 2024-03-21T06:20:03.115 INFO:teuthology.orchestra.run.smithi184.stderr:Suppressions used: 2024-03-21T06:20:03.115 INFO:teuthology.orchestra.run.smithi184.stderr: count bytes template 2024-03-21T06:20:03.115 INFO:teuthology.orchestra.run.smithi184.stderr: 1 8 MallocExtension::Register 2024-03-21T06:20:03.115 INFO:teuthology.orchestra.run.smithi184.stderr:----------------------------------------------------- 2024-03-21T06:20:03.115 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-03-21T06:20:03.178 INFO:teuthology.orchestra.run.smithi184.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-21T06:20:03.179 INFO:teuthology.orchestra.run.smithi184.stderr:INFO 2024-03-21 06:20:03,180 seastar - Reactor backend: linux-aio 2024-03-21T06:20:03.207 INFO:teuthology.orchestra.run.smithi184.stderr:==28493==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-21T06:20:06.554 INFO:teuthology.orchestra.run.smithi184.stderr:----------------------------------------------------- 2024-03-21T06:20:06.554 INFO:teuthology.orchestra.run.smithi184.stderr:Suppressions used: 2024-03-21T06:20:06.554 INFO:teuthology.orchestra.run.smithi184.stderr: count bytes template 2024-03-21T06:20:06.554 INFO:teuthology.orchestra.run.smithi184.stderr: 1 8 MallocExtension::Register 2024-03-21T06:20:06.554 INFO:teuthology.orchestra.run.smithi184.stderr:----------------------------------------------------- 2024-03-21T06:20:06.554 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-03-21T06:20:06.554 INFO:teuthology.orchestra.run.smithi184.stdout:created object store /var/lib/ceph/osd/ceph-2 for osd.2 fsid e880df84-2d7c-4386-b5e9-f3879b9ab9d5 2024-03-21T06:20:06.595 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-21T06:20:06.622 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-21T06:20:07.158 INFO:teuthology.orchestra.run.smithi184.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-21T06:20:07.159 INFO:teuthology.orchestra.run.smithi184.stderr:INFO 2024-03-21 06:20:07,159 seastar - Reactor backend: linux-aio 2024-03-21T06:20:07.179 INFO:teuthology.orchestra.run.smithi184.stderr:==29303==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-21T06:20:07.227 INFO:teuthology.orchestra.run.smithi184.stderr:INFO 2024-03-21 06:20:07,228 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-03-21T06:20:07.227 INFO:teuthology.orchestra.run.smithi184.stderr:INFO 2024-03-21 06:20:07,228 [shard 0:main] osd - get_early_config: set --cpuset 3-5 2024-03-21T06:20:07.228 INFO:teuthology.orchestra.run.smithi184.stderr:DEBUG 2024-03-21 06:20:07,228 [shard 0:main] seastar - reactor::drain 2024-03-21T06:20:07.708 INFO:teuthology.orchestra.run.smithi184.stderr:----------------------------------------------------- 2024-03-21T06:20:07.708 INFO:teuthology.orchestra.run.smithi184.stderr:Suppressions used: 2024-03-21T06:20:07.708 INFO:teuthology.orchestra.run.smithi184.stderr: count bytes template 2024-03-21T06:20:07.708 INFO:teuthology.orchestra.run.smithi184.stderr: 1 8 MallocExtension::Register 2024-03-21T06:20:07.708 INFO:teuthology.orchestra.run.smithi184.stderr:----------------------------------------------------- 2024-03-21T06:20:07.709 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-03-21T06:20:07.768 INFO:teuthology.orchestra.run.smithi184.stderr:WARNING: debug mode. Not for benchmarking or production 2024-03-21T06:20:07.770 INFO:teuthology.orchestra.run.smithi184.stderr:INFO 2024-03-21 06:20:07,771 seastar - Reactor backend: linux-aio 2024-03-21T06:20:07.798 INFO:teuthology.orchestra.run.smithi184.stderr:==29301==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-03-21T06:20:10.893 INFO:teuthology.orchestra.run.smithi184.stderr:----------------------------------------------------- 2024-03-21T06:20:10.893 INFO:teuthology.orchestra.run.smithi184.stderr:Suppressions used: 2024-03-21T06:20:10.893 INFO:teuthology.orchestra.run.smithi184.stderr: count bytes template 2024-03-21T06:20:10.893 INFO:teuthology.orchestra.run.smithi184.stderr: 1 8 MallocExtension::Register 2024-03-21T06:20:10.893 INFO:teuthology.orchestra.run.smithi184.stderr:----------------------------------------------------- 2024-03-21T06:20:10.893 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-03-21T06:20:10.893 INFO:teuthology.orchestra.run.smithi184.stdout:created object store /var/lib/ceph/osd/ceph-3 for osd.3 fsid e880df84-2d7c-4386-b5e9-f3879b9ab9d5 2024-03-21T06:20:10.935 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-21T06:20:10.962 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-21T06:20:10.962 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:20:10.962 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-21T06:20:10.995 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:20:10.995 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-21T06:20:11.065 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-21T06:20:11.066 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-21T06:20:11.092 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-21T06:20:11.092 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-21T06:20:11.158 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-21T06:20:11.158 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-21T06:20:11.224 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:20:11.225 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-21T06:20:11.243 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-21T06:20:11.243 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-21T06:20:11.280 INFO:tasks.ceph:Adding keys to all mons... 2024-03-21T06:20:11.280 DEBUG:teuthology.orchestra.run.smithi096:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-21T06:20:11.316 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.0] 2024-03-21T06:20:11.316 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCI0ftl4aT+OBAARux7LzceiBzCuMytjnrqag== 2024-03-21T06:20:11.316 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.1] 2024-03-21T06:20:11.316 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCN0ftl6gt7ERAAbSEE/Xzg1lfSEwo7vjSDzA== 2024-03-21T06:20:11.316 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr.x] 2024-03-21T06:20:11.316 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCG0ftlZrL6BBAAIWYJ3ZPB7zDXHUA2JtnaIg== 2024-03-21T06:20:11.316 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.2] 2024-03-21T06:20:11.316 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCT0ftl1Ya/DxAAqua4R21g/bkpcdWp+Aa23A== 2024-03-21T06:20:11.316 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.3] 2024-03-21T06:20:11.316 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCX0ftl4+nwMhAAg4gQqy9KKquSH5qg8JpiFg== 2024-03-21T06:20:11.316 INFO:teuthology.orchestra.run.smithi096.stdout:[client.0] 2024-03-21T06:20:11.316 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCG0ftlwwYVCBAAbNPuh9gXeobDO1CCo01wuw== 2024-03-21T06:20:11.317 INFO:teuthology.orchestra.run.smithi096.stdout:[client.1] 2024-03-21T06:20:11.317 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCG0ftlT5rdCxAAMhb4q9jLxAmy/w8N6zf4pQ== 2024-03-21T06:20:11.317 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-21T06:20:11.423 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-21T06:20:11.482 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-21T06:20:11.630 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-21T06:20:11.687 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-21T06:20:11.746 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-21T06:20:11.808 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-21T06:20:11.869 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-21T06:20:11.869 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-21T06:20:11.898 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-21T06:20:12.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.005+0000 7fd7494d79c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:12.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.005+0000 7fd7494d79c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:12.019 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-21T06:20:12.045 DEBUG:teuthology.orchestra.run.smithi096:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-21T06:20:12.099 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-21T06:20:12.100 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-21T06:20:12.100 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-21T06:20:12.141 INFO:tasks.ceph.mon.a:Started 2024-03-21T06:20:12.141 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-21T06:20:12.142 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-21T06:20:12.142 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-21T06:20:12.144 INFO:tasks.ceph.mgr.x:Started 2024-03-21T06:20:12.144 INFO:tasks.ceph:Setting crush tunables to default 2024-03-21T06:20:12.145 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd crush tunables default 2024-03-21T06:20:12.220 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-03-21T06:20:12.220+0000 7f13538579c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:12.220 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-03-21T06:20:12.220+0000 7f13538579c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:12.220 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-03-21T06:20:12.220+0000 7f13538579c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:12.236 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:12.235+0000 7fd10d7fdf00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:12.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.247+0000 7ff0ca406640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:12.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.253+0000 7ff0ca406640 1 Processor -- start 2024-03-21T06:20:12.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.254+0000 7ff0ca406640 1 -- start start 2024-03-21T06:20:12.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.254+0000 7ff0ca406640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff0c409cda0 0x7ff0c4139450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:12.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.254+0000 7ff0ca406640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c406e410 con 0x7ff0c409b360 2024-03-21T06:20:12.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.254+0000 7ff0ca406640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c409b770 con 0x7ff0c409cda0 2024-03-21T06:20:12.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.254+0000 7ff0c37fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff0c409cda0 0x7ff0c4139450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:12.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.254+0000 7ff0c3fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff0c409b360 0x7ff0c41315d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:60202/0 (socket says 172.21.15.96:60202) 2024-03-21T06:20:12.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.254+0000 7ff0c3fff640 1 -- 172.21.15.96:0/2602320686 learned_addr learned my addr 172.21.15.96:0/2602320686 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:12.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.254+0000 7ff0c37fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff0c409cda0 0x7ff0c4139450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37526/0 (socket says 172.21.15.96:37526) 2024-03-21T06:20:12.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.254+0000 7ff0c37fe640 1 -- 172.21.15.96:0/2602320686 >> v2:172.21.15.96:3300/0 conn(0x7ff0c409cda0 msgr2=0x7ff0c4139450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-21T06:20:12.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.255+0000 7ff0c37fe640 1 -- 172.21.15.96:0/2602320686 >> v2:172.21.15.96:3300/0 conn(0x7ff0c409cda0 msgr2=0x7ff0c4139450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-21T06:20:12.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.255+0000 7ff0c37fe640 1 --2- 172.21.15.96:0/2602320686 >> v2:172.21.15.96:3300/0 conn(0x7ff0c409cda0 0x7ff0c4139450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-21T06:20:12.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.255+0000 7ff0c37fe640 1 --2- 172.21.15.96:0/2602320686 >> v2:172.21.15.96:3300/0 conn(0x7ff0c409cda0 0x7ff0c4139450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T06:20:12.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.276+0000 7ff0c2ffd640 1 -- 172.21.15.96:0/2602320686 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3005313996 0 0) 0x7ff0c406e410 con 0x7ff0c409b360 2024-03-21T06:20:12.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.276+0000 7ff0c2ffd640 1 -- 172.21.15.96:0/2602320686 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0ac0034f0 con 0x7ff0c409b360 2024-03-21T06:20:12.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.277+0000 7ff0c2ffd640 1 -- 172.21.15.96:0/2602320686 <== mon.0 v1:172.21.15.96:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 468552098 0 0) 0x7ff0ac0034f0 con 0x7ff0c409b360 2024-03-21T06:20:12.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.277+0000 7ff0c2ffd640 1 -- 172.21.15.96:0/2602320686 >> v2:172.21.15.96:3300/0 conn(0x7ff0c409cda0 msgr2=0x7ff0c4139450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-21T06:20:12.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.277+0000 7ff0c2ffd640 1 --2- 172.21.15.96:0/2602320686 >> v2:172.21.15.96:3300/0 conn(0x7ff0c409cda0 0x7ff0c4139450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:12.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.277+0000 7ff0c2ffd640 1 -- 172.21.15.96:0/2602320686 --> v1:172.21.15.96:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c409d1b0 con 0x7ff0c409b360 2024-03-21T06:20:12.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.277+0000 7ff0c2ffd640 1 -- 172.21.15.96:0/2602320686 <== mon.0 v1:172.21.15.96:6789/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (unknown 1270177681 0 0) 0x7ff0b40015a0 con 0x7ff0c409b360 2024-03-21T06:20:12.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.277+0000 7ff0c2ffd640 1 -- 172.21.15.96:0/2602320686 >> v1:172.21.15.96:6789/0 conn(0x7ff0c409b360 legacy=0x7ff0c41315d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:12.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.277+0000 7ff0c2ffd640 1 --2- 172.21.15.96:0/2602320686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ac003940 0x7ff0ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:12.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.277+0000 7ff0c2ffd640 1 -- 172.21.15.96:0/2602320686 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c409b770 con 0x7ff0ac003940 2024-03-21T06:20:12.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.277+0000 7ff0c3fff640 1 --2- 172.21.15.96:0/2602320686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ac003940 0x7ff0ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:12.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.277+0000 7ff0c3fff640 1 -- 172.21.15.96:0/2602320686 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac0034f0 con 0x7ff0ac003940 2024-03-21T06:20:12.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.279+0000 7ff0c3fff640 1 --2- 172.21.15.96:0/2602320686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ac003940 0x7ff0ac003d90 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b0dc6c85ecfdd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:12.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.279+0000 7ff0c2ffd640 1 -- 172.21.15.96:0/2602320686 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff0b4007c30 con 0x7ff0ac003940 2024-03-21T06:20:12.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.279+0000 7ff0c2ffd640 1 -- 172.21.15.96:0/2602320686 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff0b40058a0 con 0x7ff0ac003940 2024-03-21T06:20:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.279+0000 7ff0c2ffd640 1 -- 172.21.15.96:0/2602320686 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff0b4005b40 con 0x7ff0ac003940 2024-03-21T06:20:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.279+0000 7ff0ca406640 1 -- 172.21.15.96:0/2602320686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ac003940 msgr2=0x7ff0ac003d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.279+0000 7ff0ca406640 1 --2- 172.21.15.96:0/2602320686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ac003940 0x7ff0ac003d90 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.279+0000 7ff0ca406640 1 -- 172.21.15.96:0/2602320686 shutdown_connections 2024-03-21T06:20:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.279+0000 7ff0ca406640 1 --2- 172.21.15.96:0/2602320686 >> v2:172.21.15.96:3300/0 conn(0x7ff0c409cda0 0x7ff0c4139450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.279+0000 7ff0ca406640 1 --2- 172.21.15.96:0/2602320686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ac003940 0x7ff0ac003d90 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.279+0000 7ff0ca406640 1 -- 172.21.15.96:0/2602320686 >> 172.21.15.96:0/2602320686 conn(0x7ff0c40a42c0 msgr2=0x7ff0c40a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.280+0000 7ff0ca406640 1 -- 172.21.15.96:0/2602320686 shutdown_connections 2024-03-21T06:20:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.280+0000 7ff0ca406640 1 -- 172.21.15.96:0/2602320686 wait complete. 2024-03-21T06:20:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.280+0000 7ff0ca406640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:12.282 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:12.280+0000 7fd10d7fdf00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:12.282 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:12.280+0000 7fd10d7fdf00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:12.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.281+0000 7ff0ca406640 1 Processor -- start 2024-03-21T06:20:12.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.281+0000 7ff0ca406640 1 -- start start 2024-03-21T06:20:12.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.282+0000 7ff0ca406640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ac003940 0x7ff0c412f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:12.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.282+0000 7ff0ca406640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c4138cb0 con 0x7ff0ac003940 2024-03-21T06:20:12.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.282+0000 7ff0c3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ac003940 0x7ff0c412f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:12.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.282+0000 7ff0c3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ac003940 0x7ff0c412f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37558/0 (socket says 172.21.15.96:37558) 2024-03-21T06:20:12.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.282+0000 7ff0c3fff640 1 -- 172.21.15.96:0/141197186 learned_addr learned my addr 172.21.15.96:0/141197186 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:12.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.282+0000 7ff0c3fff640 1 -- 172.21.15.96:0/141197186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c411cb70 con 0x7ff0ac003940 2024-03-21T06:20:12.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.282+0000 7ff0c3fff640 1 --2- 172.21.15.96:0/141197186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ac003940 0x7ff0c412f590 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:12.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.282+0000 7ff0c0ff9640 1 -- 172.21.15.96:0/141197186 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff0b4001560 con 0x7ff0ac003940 2024-03-21T06:20:12.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.282+0000 7ff0c0ff9640 1 -- 172.21.15.96:0/141197186 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7ff0b4007a50 con 0x7ff0ac003940 2024-03-21T06:20:12.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.282+0000 7ff0ca406640 1 -- 172.21.15.96:0/141197186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c411d520 con 0x7ff0ac003940 2024-03-21T06:20:12.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.282+0000 7ff0c0ff9640 1 -- 172.21.15.96:0/141197186 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7ff0b4000ed0 con 0x7ff0ac003940 2024-03-21T06:20:12.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.282+0000 7ff0ca406640 1 -- 172.21.15.96:0/141197186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c411ec30 con 0x7ff0ac003940 2024-03-21T06:20:12.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.282+0000 7ff0c0ff9640 1 -- 172.21.15.96:0/141197186 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7ff0b40167b0 con 0x7ff0ac003940 2024-03-21T06:20:12.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.282+0000 7ff0c0ff9640 1 -- 172.21.15.96:0/141197186 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (crc 0 0 0) 0x7ff0b4015ce0 con 0x7ff0ac003940 2024-03-21T06:20:12.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.283+0000 7ff0ca406640 1 -- 172.21.15.96:0/141197186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c409ba50 con 0x7ff0ac003940 2024-03-21T06:20:12.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.285+0000 7ff0c0ff9640 1 -- 172.21.15.96:0/141197186 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (crc 0 0 0) 0x7ff0b40153b0 con 0x7ff0ac003940 2024-03-21T06:20:12.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.348+0000 7ff0ca406640 1 -- 172.21.15.96:0/141197186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff0c4062f10 con 0x7ff0ac003940 2024-03-21T06:20:12.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.349+0000 7ff0c0ff9640 1 -- 172.21.15.96:0/141197186 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (crc 0 0 0) 0x7ff0b4015550 con 0x7ff0ac003940 2024-03-21T06:20:12.350 INFO:teuthology.orchestra.run.smithi096.stderr:adjusted tunables profile to default 2024-03-21T06:20:12.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.350+0000 7ff0ca406640 1 -- 172.21.15.96:0/141197186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ac003940 msgr2=0x7ff0c412f590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:12.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.350+0000 7ff0ca406640 1 --2- 172.21.15.96:0/141197186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ac003940 0x7ff0c412f590 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:12.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.350+0000 7ff0ca406640 1 -- 172.21.15.96:0/141197186 shutdown_connections 2024-03-21T06:20:12.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.350+0000 7ff0ca406640 1 --2- 172.21.15.96:0/141197186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0ac003940 0x7ff0c412f590 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:12.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.350+0000 7ff0ca406640 1 -- 172.21.15.96:0/141197186 >> 172.21.15.96:0/141197186 conn(0x7ff0c40a42c0 msgr2=0x7ff0c40a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:12.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.350+0000 7ff0ca406640 1 -- 172.21.15.96:0/141197186 shutdown_connections 2024-03-21T06:20:12.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.350+0000 7ff0ca406640 1 -- 172.21.15.96:0/141197186 wait complete. 2024-03-21T06:20:12.361 INFO:tasks.ceph:check_enable_crimson: True 2024-03-21T06:20:12.361 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2024-03-21T06:20:12.362 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2024-03-21T06:20:12.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.459+0000 7f36ccd5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:12.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.464+0000 7f36ccd5c640 1 Processor -- start 2024-03-21T06:20:12.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.465+0000 7f36ccd5c640 1 -- start start 2024-03-21T06:20:12.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.465+0000 7f36ccd5c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f36c8130b70 0x7f36c8135990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:12.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.465+0000 7f36ccd5c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c8099dc0 con 0x7f36c809b5d0 2024-03-21T06:20:12.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.465+0000 7f36ccd5c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c809bae0 con 0x7f36c8130b70 2024-03-21T06:20:12.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.465+0000 7f36c659c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f36c809b5d0 0x7f36c8130460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:60208/0 (socket says 172.21.15.96:60208) 2024-03-21T06:20:12.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.465+0000 7f36c659c640 1 -- 172.21.15.96:0/3265821716 learned_addr learned my addr 172.21.15.96:0/3265821716 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:12.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.465+0000 7f36c5d9b640 1 --2- 172.21.15.96:0/3265821716 >> v2:172.21.15.96:3300/0 conn(0x7f36c8130b70 0x7f36c8135990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:12.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.466+0000 7f36c559a640 1 -- 172.21.15.96:0/3265821716 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4257066510 0 0) 0x7f36c8099dc0 con 0x7f36c809b5d0 2024-03-21T06:20:12.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.466+0000 7f36c559a640 1 -- 172.21.15.96:0/3265821716 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36b0004600 con 0x7f36c809b5d0 2024-03-21T06:20:12.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.466+0000 7f36c5d9b640 1 -- 172.21.15.96:0/3265821716 >> v1:172.21.15.96:6789/0 conn(0x7f36c809b5d0 legacy=0x7f36c8130460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:12.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.466+0000 7f36c5d9b640 1 -- 172.21.15.96:0/3265821716 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c809c0f0 con 0x7f36c8130b70 2024-03-21T06:20:12.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.466+0000 7f36c5d9b640 1 --2- 172.21.15.96:0/3265821716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36c8130b70 0x7f36c8135990 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f075a7bbba6266 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:12.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.466+0000 7f36c559a640 1 -- 172.21.15.96:0/3265821716 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f36b8009ce0 con 0x7f36c8130b70 2024-03-21T06:20:12.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.466+0000 7f36c559a640 1 -- 172.21.15.96:0/3265821716 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f36b8009e40 con 0x7f36c8130b70 2024-03-21T06:20:12.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.466+0000 7f36c559a640 1 -- 172.21.15.96:0/3265821716 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f36b800e360 con 0x7f36c8130b70 2024-03-21T06:20:12.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.467+0000 7f36ccd5c640 1 -- 172.21.15.96:0/3265821716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36c8130b70 msgr2=0x7f36c8135990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:12.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.467+0000 7f36ccd5c640 1 --2- 172.21.15.96:0/3265821716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36c8130b70 0x7f36c8135990 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:12.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.467+0000 7f36ccd5c640 1 -- 172.21.15.96:0/3265821716 shutdown_connections 2024-03-21T06:20:12.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.467+0000 7f36ccd5c640 1 --2- 172.21.15.96:0/3265821716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36c8130b70 0x7f36c8135990 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:12.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.467+0000 7f36ccd5c640 1 -- 172.21.15.96:0/3265821716 >> 172.21.15.96:0/3265821716 conn(0x7f36c809fdb0 msgr2=0x7f36c80a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:12.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.467+0000 7f36ccd5c640 1 -- 172.21.15.96:0/3265821716 shutdown_connections 2024-03-21T06:20:12.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.467+0000 7f36ccd5c640 1 -- 172.21.15.96:0/3265821716 wait complete. 2024-03-21T06:20:12.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.467+0000 7f36ccd5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:12.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.468+0000 7f36ccd5c640 1 Processor -- start 2024-03-21T06:20:12.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.468+0000 7f36ccd5c640 1 -- start start 2024-03-21T06:20:12.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.469+0000 7f36ccd5c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36c809b5d0 0x7f36c81c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:12.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.469+0000 7f36ccd5c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c8128350 con 0x7f36c809b5d0 2024-03-21T06:20:12.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.469+0000 7f36c659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36c809b5d0 0x7f36c81c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:12.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.469+0000 7f36c659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36c809b5d0 0x7f36c81c3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37584/0 (socket says 172.21.15.96:37584) 2024-03-21T06:20:12.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.469+0000 7f36c659c640 1 -- 172.21.15.96:0/1418367963 learned_addr learned my addr 172.21.15.96:0/1418367963 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:12.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.469+0000 7f36c659c640 1 -- 172.21.15.96:0/1418367963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c8129200 con 0x7f36c809b5d0 2024-03-21T06:20:12.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.469+0000 7f36c659c640 1 --2- 172.21.15.96:0/1418367963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36c809b5d0 0x7f36c81c3b50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:12.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.469+0000 7f36b6ffd640 1 -- 172.21.15.96:0/1418367963 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f36bc0059d0 con 0x7f36c809b5d0 2024-03-21T06:20:12.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.469+0000 7f36b6ffd640 1 -- 172.21.15.96:0/1418367963 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f36bc005b30 con 0x7f36c809b5d0 2024-03-21T06:20:12.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.469+0000 7f36ccd5c640 1 -- 172.21.15.96:0/1418367963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c8129c40 con 0x7f36c809b5d0 2024-03-21T06:20:12.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.469+0000 7f36b6ffd640 1 -- 172.21.15.96:0/1418367963 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f36bc005dd0 con 0x7f36c809b5d0 2024-03-21T06:20:12.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.469+0000 7f36ccd5c640 1 -- 172.21.15.96:0/1418367963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c8129550 con 0x7f36c809b5d0 2024-03-21T06:20:12.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.469+0000 7f36b6ffd640 1 -- 172.21.15.96:0/1418367963 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f36bc007440 con 0x7f36c809b5d0 2024-03-21T06:20:12.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.469+0000 7f36b6ffd640 1 -- 172.21.15.96:0/1418367963 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (crc 0 0 0) 0x7f36bc00f930 con 0x7f36c809b5d0 2024-03-21T06:20:12.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.470+0000 7f36ccd5c640 1 -- 172.21.15.96:0/1418367963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c8062f10 con 0x7f36c809b5d0 2024-03-21T06:20:12.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.472+0000 7f36b6ffd640 1 -- 172.21.15.96:0/1418367963 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (crc 0 0 0) 0x7f36bc00fb90 con 0x7f36c809b5d0 2024-03-21T06:20:12.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:12.546+0000 7f36ccd5c640 1 -- 172.21.15.96:0/1418367963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f36c8130240 con 0x7f36c809b5d0 2024-03-21T06:20:12.547 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-03-21T06:20:12.546+0000 7f134b7f1640 -1 WARNING: experimental feature 'crimson' is enabled 2024-03-21T06:20:12.547 INFO:tasks.ceph.mon.a.smithi096.stderr:Please be aware that this feature is experimental, untested, 2024-03-21T06:20:12.547 INFO:tasks.ceph.mon.a.smithi096.stderr:unsupported, and may result in data corruption, data loss, 2024-03-21T06:20:12.547 INFO:tasks.ceph.mon.a.smithi096.stderr:and/or irreparable damage to your cluster. Do not use 2024-03-21T06:20:12.547 INFO:tasks.ceph.mon.a.smithi096.stderr:feature with important data. 2024-03-21T06:20:12.547 INFO:tasks.ceph.mon.a.smithi096.stderr: 2024-03-21T06:20:12.758 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:12.758+0000 7fd10d7fdf00 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T06:20:12.812 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:12.811+0000 7fd10d7fdf00 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T06:20:13.336 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:13.336+0000 7fd10d7fdf00 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T06:20:13.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:13.350+0000 7f36b6ffd640 1 -- 172.21.15.96:0/1418367963 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true}]=0 v3) v1 ==== 99+0+0 (crc 0 0 0) 0x7f36bc00fd30 con 0x7f36c809b5d0 2024-03-21T06:20:13.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:13.351+0000 7f36ccd5c640 1 -- 172.21.15.96:0/1418367963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36c809b5d0 msgr2=0x7f36c81c3b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:13.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:13.351+0000 7f36ccd5c640 1 --2- 172.21.15.96:0/1418367963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36c809b5d0 0x7f36c81c3b50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:13.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:13.351+0000 7f36ccd5c640 1 -- 172.21.15.96:0/1418367963 shutdown_connections 2024-03-21T06:20:13.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:13.351+0000 7f36ccd5c640 1 --2- 172.21.15.96:0/1418367963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36c809b5d0 0x7f36c81c3b50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:13.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:13.351+0000 7f36ccd5c640 1 -- 172.21.15.96:0/1418367963 >> 172.21.15.96:0/1418367963 conn(0x7f36c809fdb0 msgr2=0x7f36c80a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:13.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:13.352+0000 7f36ccd5c640 1 -- 172.21.15.96:0/1418367963 shutdown_connections 2024-03-21T06:20:13.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:13.352+0000 7f36ccd5c640 1 -- 172.21.15.96:0/1418367963 wait complete. 2024-03-21T06:20:13.366 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-21T06:20:13.366 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:20:13.367 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-21T06:20:13.389 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:13.389+0000 7fd10d7fdf00 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T06:20:13.398 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:20:13.398 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-21T06:20:13.433 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:13.433+0000 7fd10d7fdf00 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T06:20:13.467 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-21T06:20:13.467 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-21T06:20:13.487 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:13.487+0000 7fd10d7fdf00 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T06:20:13.495 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-03-21T06:20:13.495 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-21T06:20:13.533 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:13.532+0000 7fd10d7fdf00 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T06:20:13.559 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new ffab25be-1f6f-450d-a8b4-da7f86f3eaeb 0 2024-03-21T06:20:13.655 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:13.654+0000 7fd10d7fdf00 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T06:20:13.705 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.699+0000 7f83e759c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:13.705 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.705+0000 7f83e759c640 1 Processor -- start 2024-03-21T06:20:13.705 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.705+0000 7f83e759c640 1 -- start start 2024-03-21T06:20:13.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.705+0000 7f83e759c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f83e0119040 0x7f83e0119450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:13.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.705+0000 7f83e759c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e0113910 con 0x7f83e0112df0 2024-03-21T06:20:13.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.705+0000 7f83e759c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e0113aa0 con 0x7f83e0119040 2024-03-21T06:20:13.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.706+0000 7f83e5338640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f83e0119040 0x7f83e0119450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:13.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.706+0000 7f83e5338640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f83e0119040 0x7f83e0119450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.184:40980/0 (socket says 172.21.15.184:40980) 2024-03-21T06:20:13.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.706+0000 7f83e5338640 1 -- 172.21.15.184:0/488978345 learned_addr learned my addr 172.21.15.184:0/488978345 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-03-21T06:20:13.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.706+0000 7f83e4b37640 1 --1- 172.21.15.184:0/488978345 >> v1:172.21.15.96:6789/0 conn(0x7f83e0112df0 0x7f83e0113200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.184:37586/0 (socket says 172.21.15.184:37586) 2024-03-21T06:20:13.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.706+0000 7f83e5338640 1 -- 172.21.15.184:0/488978345 >> v1:172.21.15.96:6789/0 conn(0x7f83e0112df0 legacy=0x7f83e0113200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:13.707 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.706+0000 7f83e5338640 1 -- 172.21.15.184:0/488978345 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e0113c10 con 0x7f83e0119040 2024-03-21T06:20:13.708 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.706+0000 7f83e5338640 1 --2- 172.21.15.184:0/488978345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0119040 0x7f83e0119450 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fca1505f92e84c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:13.708 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.707+0000 7f83d7fff640 1 -- 172.21.15.184:0/488978345 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f83d0009dd0 con 0x7f83e0119040 2024-03-21T06:20:13.708 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.707+0000 7f83d7fff640 1 -- 172.21.15.184:0/488978345 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f83d000e230 con 0x7f83e0119040 2024-03-21T06:20:13.708 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.707+0000 7f83d7fff640 1 -- 172.21.15.184:0/488978345 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f83d000e4d0 con 0x7f83e0119040 2024-03-21T06:20:13.710 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.707+0000 7f83e759c640 1 -- 172.21.15.184:0/488978345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0119040 msgr2=0x7f83e0119450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:13.710 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.707+0000 7f83e759c640 1 --2- 172.21.15.184:0/488978345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0119040 0x7f83e0119450 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:13.710 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.707+0000 7f83e759c640 1 -- 172.21.15.184:0/488978345 shutdown_connections 2024-03-21T06:20:13.710 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.707+0000 7f83e759c640 1 --2- 172.21.15.184:0/488978345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0119040 0x7f83e0119450 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:13.710 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.707+0000 7f83e759c640 1 -- 172.21.15.184:0/488978345 >> 172.21.15.184:0/488978345 conn(0x7f83e006d0e0 msgr2=0x7f83e006d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:13.711 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.707+0000 7f83e759c640 1 -- 172.21.15.184:0/488978345 shutdown_connections 2024-03-21T06:20:13.711 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.707+0000 7f83e759c640 1 -- 172.21.15.184:0/488978345 wait complete. 2024-03-21T06:20:13.711 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.708+0000 7f83e759c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:13.711 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.708+0000 7f83e759c640 1 Processor -- start 2024-03-21T06:20:13.711 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.708+0000 7f83e759c640 1 -- start start 2024-03-21T06:20:13.711 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.708+0000 7f83e759c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0112df0 0x7f83e01a7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:13.711 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.708+0000 7f83e759c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e01a8230 con 0x7f83e0112df0 2024-03-21T06:20:13.711 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.709+0000 7f83e5338640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0112df0 0x7f83e01a7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:13.711 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.709+0000 7f83e5338640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0112df0 0x7f83e01a7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.184:40984/0 (socket says 172.21.15.184:40984) 2024-03-21T06:20:13.711 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.709+0000 7f83e5338640 1 -- 172.21.15.184:0/2500774328 learned_addr learned my addr 172.21.15.184:0/2500774328 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-03-21T06:20:13.711 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.709+0000 7f83e5338640 1 -- 172.21.15.184:0/2500774328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e01a8370 con 0x7f83e0112df0 2024-03-21T06:20:13.711 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.709+0000 7f83e5338640 1 --2- 172.21.15.184:0/2500774328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0112df0 0x7f83e01a7c60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:13.711 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.709+0000 7f83d5ffb640 1 -- 172.21.15.184:0/2500774328 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f83d0014540 con 0x7f83e0112df0 2024-03-21T06:20:13.711 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.709+0000 7f83d5ffb640 1 -- 172.21.15.184:0/2500774328 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f83d00146a0 con 0x7f83e0112df0 2024-03-21T06:20:13.712 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.709+0000 7f83e759c640 1 -- 172.21.15.184:0/2500774328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e01a85c0 con 0x7f83e0112df0 2024-03-21T06:20:13.712 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.709+0000 7f83d5ffb640 1 -- 172.21.15.184:0/2500774328 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f83d0014940 con 0x7f83e0112df0 2024-03-21T06:20:13.712 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.710+0000 7f83e759c640 1 -- 172.21.15.184:0/2500774328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e01ac450 con 0x7f83e0112df0 2024-03-21T06:20:13.712 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.710+0000 7f83d5ffb640 1 -- 172.21.15.184:0/2500774328 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f83d0013400 con 0x7f83e0112df0 2024-03-21T06:20:13.712 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.710+0000 7f83d5ffb640 1 -- 172.21.15.184:0/2500774328 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 747+0+0 (crc 0 0 0) 0x7f83d0019040 con 0x7f83e0112df0 2024-03-21T06:20:13.712 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.711+0000 7f83e759c640 1 -- 172.21.15.184:0/2500774328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e0073c60 con 0x7f83e0112df0 2024-03-21T06:20:13.713 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.713+0000 7f83d5ffb640 1 -- 172.21.15.184:0/2500774328 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (crc 0 0 0) 0x7f83d0012390 con 0x7f83e0112df0 2024-03-21T06:20:13.730 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:13.729+0000 7fd10d7fdf00 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T06:20:13.777 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:13.777+0000 7fd10d7fdf00 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T06:20:13.786 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.785+0000 7f83e759c640 1 -- 172.21.15.184:0/2500774328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ffab25be-1f6f-450d-a8b4-da7f86f3eaeb", "id": 0} v 0) v1 -- 0x7f83e0119230 con 0x7f83e0112df0 2024-03-21T06:20:13.787 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.787+0000 7f83d5ffb640 1 -- 172.21.15.184:0/2500774328 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ffab25be-1f6f-450d-a8b4-da7f86f3eaeb", "id": 0}]=0 v4) v1 ==== 112+0+2 (crc 0 0 0) 0x7f83d0013930 con 0x7f83e0112df0 2024-03-21T06:20:13.787 INFO:teuthology.orchestra.run.smithi184.stdout:0 2024-03-21T06:20:13.788 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.788+0000 7f83e759c640 1 -- 172.21.15.184:0/2500774328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0112df0 msgr2=0x7f83e01a7c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:13.789 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.788+0000 7f83e759c640 1 --2- 172.21.15.184:0/2500774328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0112df0 0x7f83e01a7c60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:13.789 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.788+0000 7f83e759c640 1 -- 172.21.15.184:0/2500774328 shutdown_connections 2024-03-21T06:20:13.789 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.788+0000 7f83e759c640 1 --2- 172.21.15.184:0/2500774328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e0112df0 0x7f83e01a7c60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:13.789 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.788+0000 7f83e759c640 1 -- 172.21.15.184:0/2500774328 >> 172.21.15.184:0/2500774328 conn(0x7f83e006d0e0 msgr2=0x7f83e0114c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:13.789 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.788+0000 7f83e759c640 1 -- 172.21.15.184:0/2500774328 shutdown_connections 2024-03-21T06:20:13.789 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.788+0000 7f83e759c640 1 -- 172.21.15.184:0/2500774328 wait complete. 2024-03-21T06:20:13.799 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 3266cd80-6286-4a69-933a-00cd48d67487 1 2024-03-21T06:20:13.834 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:13.833+0000 7fd10d7fdf00 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T06:20:13.885 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:13.884+0000 7fd10d7fdf00 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T06:20:13.905 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.899+0000 7f0ea5c0f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:13.906 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.905+0000 7f0ea5c0f640 1 Processor -- start 2024-03-21T06:20:13.907 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.905+0000 7f0ea5c0f640 1 -- start start 2024-03-21T06:20:13.907 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.905+0000 7f0ea5c0f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0ea011bd20 0x7f0ea011e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:13.907 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.905+0000 7f0ea5c0f640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea011e710 con 0x7f0ea0119040 2024-03-21T06:20:13.907 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.905+0000 7f0ea5c0f640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea011f8b0 con 0x7f0ea011bd20 2024-03-21T06:20:13.907 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.906+0000 7f0e96ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0ea011bd20 0x7f0ea011e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:13.907 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.906+0000 7f0e9f7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f0ea0119040 0x7f0ea0119450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.184:37594/0 (socket says 172.21.15.184:37594) 2024-03-21T06:20:13.907 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.906+0000 7f0e9f7fe640 1 -- 172.21.15.184:0/631674843 learned_addr learned my addr 172.21.15.184:0/631674843 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-03-21T06:20:13.907 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.906+0000 7f0e96ffd640 1 -- 172.21.15.184:0/631674843 >> v1:172.21.15.96:6789/0 conn(0x7f0ea0119040 legacy=0x7f0ea0119450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:13.907 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.906+0000 7f0e96ffd640 1 -- 172.21.15.184:0/631674843 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea011fa20 con 0x7f0ea011bd20 2024-03-21T06:20:13.907 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.906+0000 7f0e96ffd640 1 --2- 172.21.15.184:0/631674843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea011bd20 0x7f0ea011e140 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642016935cd56e70 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:13.907 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.906+0000 7f0e9effd640 1 -- 172.21.15.184:0/631674843 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0e88009ce0 con 0x7f0ea011bd20 2024-03-21T06:20:13.907 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.906+0000 7f0e9effd640 1 -- 172.21.15.184:0/631674843 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0e88009e40 con 0x7f0ea011bd20 2024-03-21T06:20:13.908 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.906+0000 7f0e9effd640 1 -- 172.21.15.184:0/631674843 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0e8800e360 con 0x7f0ea011bd20 2024-03-21T06:20:13.910 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.907+0000 7f0ea5c0f640 1 -- 172.21.15.184:0/631674843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea011bd20 msgr2=0x7f0ea011e140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:13.910 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.907+0000 7f0ea5c0f640 1 --2- 172.21.15.184:0/631674843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea011bd20 0x7f0ea011e140 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:13.910 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.907+0000 7f0ea5c0f640 1 -- 172.21.15.184:0/631674843 shutdown_connections 2024-03-21T06:20:13.910 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.907+0000 7f0ea5c0f640 1 --2- 172.21.15.184:0/631674843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea011bd20 0x7f0ea011e140 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:13.911 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.907+0000 7f0ea5c0f640 1 -- 172.21.15.184:0/631674843 >> 172.21.15.184:0/631674843 conn(0x7f0ea006d0e0 msgr2=0x7f0ea006d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:13.911 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.907+0000 7f0ea5c0f640 1 -- 172.21.15.184:0/631674843 shutdown_connections 2024-03-21T06:20:13.911 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.907+0000 7f0ea5c0f640 1 -- 172.21.15.184:0/631674843 wait complete. 2024-03-21T06:20:13.911 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.907+0000 7f0ea5c0f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:13.911 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.908+0000 7f0ea5c0f640 1 Processor -- start 2024-03-21T06:20:13.911 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.908+0000 7f0ea5c0f640 1 -- start start 2024-03-21T06:20:13.911 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.908+0000 7f0ea5c0f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea0119040 0x7f0ea01a7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:13.911 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.908+0000 7f0ea5c0f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea01a8280 con 0x7f0ea0119040 2024-03-21T06:20:13.911 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.909+0000 7f0e9f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea0119040 0x7f0ea01a7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:13.911 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.909+0000 7f0e9f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea0119040 0x7f0ea01a7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.184:40992/0 (socket says 172.21.15.184:40992) 2024-03-21T06:20:13.911 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.909+0000 7f0e9f7fe640 1 -- 172.21.15.184:0/1575995511 learned_addr learned my addr 172.21.15.184:0/1575995511 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-03-21T06:20:13.911 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.909+0000 7f0e9f7fe640 1 -- 172.21.15.184:0/1575995511 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea01a83c0 con 0x7f0ea0119040 2024-03-21T06:20:13.911 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.909+0000 7f0e9f7fe640 1 --2- 172.21.15.184:0/1575995511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea0119040 0x7f0ea01a7cb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:13.912 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.909+0000 7f0e9cff9640 1 -- 172.21.15.184:0/1575995511 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0e900059d0 con 0x7f0ea0119040 2024-03-21T06:20:13.912 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.909+0000 7f0e9cff9640 1 -- 172.21.15.184:0/1575995511 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0e90005b30 con 0x7f0ea0119040 2024-03-21T06:20:13.912 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.909+0000 7f0ea5c0f640 1 -- 172.21.15.184:0/1575995511 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea01a8670 con 0x7f0ea0119040 2024-03-21T06:20:13.912 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.909+0000 7f0e9cff9640 1 -- 172.21.15.184:0/1575995511 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0e90005dd0 con 0x7f0ea0119040 2024-03-21T06:20:13.912 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.909+0000 7f0ea5c0f640 1 -- 172.21.15.184:0/1575995511 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea01ac4a0 con 0x7f0ea0119040 2024-03-21T06:20:13.912 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.910+0000 7f0e9cff9640 1 -- 172.21.15.184:0/1575995511 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f0e90007440 con 0x7f0ea0119040 2024-03-21T06:20:13.912 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.910+0000 7f0e9cff9640 1 -- 172.21.15.184:0/1575995511 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 862+0+0 (crc 0 0 0) 0x7f0e9000fa50 con 0x7f0ea0119040 2024-03-21T06:20:13.912 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.910+0000 7f0ea5c0f640 1 -- 172.21.15.184:0/1575995511 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea0073c60 con 0x7f0ea0119040 2024-03-21T06:20:13.913 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.913+0000 7f0e9cff9640 1 -- 172.21.15.184:0/1575995511 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (crc 0 0 0) 0x7f0e9000f230 con 0x7f0ea0119040 2024-03-21T06:20:13.985 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.985+0000 7f0ea5c0f640 1 -- 172.21.15.184:0/1575995511 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3266cd80-6286-4a69-933a-00cd48d67487", "id": 1} v 0) v1 -- 0x7f0ea011bf30 con 0x7f0ea0119040 2024-03-21T06:20:13.987 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.986+0000 7f0e9cff9640 1 -- 172.21.15.184:0/1575995511 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3266cd80-6286-4a69-933a-00cd48d67487", "id": 1}]=0 v5) v1 ==== 112+0+2 (crc 0 0 0) 0x7f0e9000fce0 con 0x7f0ea0119040 2024-03-21T06:20:13.987 INFO:teuthology.orchestra.run.smithi184.stdout:1 2024-03-21T06:20:13.988 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.987+0000 7f0ea5c0f640 1 -- 172.21.15.184:0/1575995511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea0119040 msgr2=0x7f0ea01a7cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:13.988 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.987+0000 7f0ea5c0f640 1 --2- 172.21.15.184:0/1575995511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea0119040 0x7f0ea01a7cb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:13.988 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.987+0000 7f0ea5c0f640 1 -- 172.21.15.184:0/1575995511 shutdown_connections 2024-03-21T06:20:13.988 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.987+0000 7f0ea5c0f640 1 --2- 172.21.15.184:0/1575995511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ea0119040 0x7f0ea01a7cb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:13.988 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.987+0000 7f0ea5c0f640 1 -- 172.21.15.184:0/1575995511 >> 172.21.15.184:0/1575995511 conn(0x7f0ea006d0e0 msgr2=0x7f0ea0111da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:13.988 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.987+0000 7f0ea5c0f640 1 -- 172.21.15.184:0/1575995511 shutdown_connections 2024-03-21T06:20:13.988 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:13.987+0000 7f0ea5c0f640 1 -- 172.21.15.184:0/1575995511 wait complete. 2024-03-21T06:20:13.998 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 5a8b1e5a-d4ed-4c96-b301-4173028235d6 2 2024-03-21T06:20:14.146 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.140+0000 7fbad6d5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:14.148 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.146+0000 7fbad6d5c640 1 Processor -- start 2024-03-21T06:20:14.148 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.146+0000 7fbad6d5c640 1 -- start start 2024-03-21T06:20:14.148 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.146+0000 7fbad6d5c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbad011bd20 0x7fbad011e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:14.148 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.146+0000 7fbad6d5c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad011e710 con 0x7fbad0119040 2024-03-21T06:20:14.148 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.146+0000 7fbad6d5c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad011f8b0 con 0x7fbad011bd20 2024-03-21T06:20:14.148 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.147+0000 7fbacffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbad011bd20 0x7fbad011e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:14.148 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.147+0000 7fbacffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbad011bd20 0x7fbad011e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.184:40998/0 (socket says 172.21.15.184:40998) 2024-03-21T06:20:14.148 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.147+0000 7fbad4af8640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fbad0119040 0x7fbad0119450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.184:37600/0 (socket says 172.21.15.184:37600) 2024-03-21T06:20:14.148 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.147+0000 7fbacffff640 1 -- 172.21.15.184:0/1734656990 learned_addr learned my addr 172.21.15.184:0/1734656990 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-03-21T06:20:14.149 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.147+0000 7fbacffff640 1 -- 172.21.15.184:0/1734656990 >> v1:172.21.15.96:6789/0 conn(0x7fbad0119040 legacy=0x7fbad0119450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:14.149 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.147+0000 7fbacf7fe640 1 -- 172.21.15.184:0/1734656990 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288062407 0 0) 0x7fbad011e710 con 0x7fbad0119040 2024-03-21T06:20:14.149 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.147+0000 7fbacffff640 1 -- 172.21.15.184:0/1734656990 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad011fa20 con 0x7fbad011bd20 2024-03-21T06:20:14.149 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.147+0000 7fbacffff640 1 --2- 172.21.15.184:0/1734656990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad011bd20 0x7fbad011e140 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6c45b2fcfa3468 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:14.149 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.148+0000 7fbacf7fe640 1 -- 172.21.15.184:0/1734656990 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbabc009dd0 con 0x7fbad011bd20 2024-03-21T06:20:14.149 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.148+0000 7fbacf7fe640 1 -- 172.21.15.184:0/1734656990 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fbabc00e230 con 0x7fbad011bd20 2024-03-21T06:20:14.149 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.148+0000 7fbacf7fe640 1 -- 172.21.15.184:0/1734656990 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbabc00e4d0 con 0x7fbad011bd20 2024-03-21T06:20:14.154 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.148+0000 7fbad6d5c640 1 -- 172.21.15.184:0/1734656990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad011bd20 msgr2=0x7fbad011e140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:14.154 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.148+0000 7fbad6d5c640 1 --2- 172.21.15.184:0/1734656990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad011bd20 0x7fbad011e140 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:14.154 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.148+0000 7fbad6d5c640 1 -- 172.21.15.184:0/1734656990 shutdown_connections 2024-03-21T06:20:14.154 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.148+0000 7fbad6d5c640 1 --2- 172.21.15.184:0/1734656990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad011bd20 0x7fbad011e140 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:14.154 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.148+0000 7fbad6d5c640 1 -- 172.21.15.184:0/1734656990 >> 172.21.15.184:0/1734656990 conn(0x7fbad006d0e0 msgr2=0x7fbad006d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:14.155 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.148+0000 7fbad6d5c640 1 -- 172.21.15.184:0/1734656990 shutdown_connections 2024-03-21T06:20:14.155 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.148+0000 7fbad6d5c640 1 -- 172.21.15.184:0/1734656990 wait complete. 2024-03-21T06:20:14.155 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.149+0000 7fbad6d5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:14.155 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.149+0000 7fbad6d5c640 1 Processor -- start 2024-03-21T06:20:14.155 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.149+0000 7fbad6d5c640 1 -- start start 2024-03-21T06:20:14.155 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.150+0000 7fbad6d5c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad0119040 0x7fbad01a7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:14.155 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.150+0000 7fbad6d5c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad01a8130 con 0x7fbad0119040 2024-03-21T06:20:14.155 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.150+0000 7fbad4af8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad0119040 0x7fbad01a7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:14.155 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.150+0000 7fbad4af8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad0119040 0x7fbad01a7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.184:41008/0 (socket says 172.21.15.184:41008) 2024-03-21T06:20:14.155 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.150+0000 7fbad4af8640 1 -- 172.21.15.184:0/3350075716 learned_addr learned my addr 172.21.15.184:0/3350075716 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-03-21T06:20:14.155 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.150+0000 7fbad4af8640 1 -- 172.21.15.184:0/3350075716 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad01a8270 con 0x7fbad0119040 2024-03-21T06:20:14.155 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.151+0000 7fbad4af8640 1 --2- 172.21.15.184:0/3350075716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad0119040 0x7fbad01a7b60 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:14.156 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.151+0000 7fbacd7fa640 1 -- 172.21.15.184:0/3350075716 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbac4004bd0 con 0x7fbad0119040 2024-03-21T06:20:14.156 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.151+0000 7fbacd7fa640 1 -- 172.21.15.184:0/3350075716 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fbac4004d30 con 0x7fbad0119040 2024-03-21T06:20:14.156 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.151+0000 7fbad6d5c640 1 -- 172.21.15.184:0/3350075716 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad01a8520 con 0x7fbad0119040 2024-03-21T06:20:14.156 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.151+0000 7fbacd7fa640 1 -- 172.21.15.184:0/3350075716 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fbac4007120 con 0x7fbad0119040 2024-03-21T06:20:14.156 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.151+0000 7fbad6d5c640 1 -- 172.21.15.184:0/3350075716 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad01ac260 con 0x7fbad0119040 2024-03-21T06:20:14.156 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.151+0000 7fbacd7fa640 1 -- 172.21.15.184:0/3350075716 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7fbac400c580 con 0x7fbad0119040 2024-03-21T06:20:14.156 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.151+0000 7fbacd7fa640 1 -- 172.21.15.184:0/3350075716 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 977+0+0 (crc 0 0 0) 0x7fbac400f230 con 0x7fbad0119040 2024-03-21T06:20:14.156 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.152+0000 7fbad6d5c640 1 -- 172.21.15.184:0/3350075716 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad0073c60 con 0x7fbad0119040 2024-03-21T06:20:14.157 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.154+0000 7fbacd7fa640 1 -- 172.21.15.184:0/3350075716 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (crc 0 0 0) 0x7fbac400c830 con 0x7fbad0119040 2024-03-21T06:20:14.227 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.226+0000 7fbad6d5c640 1 -- 172.21.15.184:0/3350075716 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5a8b1e5a-d4ed-4c96-b301-4173028235d6", "id": 2} v 0) v1 -- 0x7fbad011bdf0 con 0x7fbad0119040 2024-03-21T06:20:14.228 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.228+0000 7fbacd7fa640 1 -- 172.21.15.184:0/3350075716 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5a8b1e5a-d4ed-4c96-b301-4173028235d6", "id": 2}]=0 v6) v1 ==== 112+0+2 (crc 0 0 0) 0x7fbac400c9d0 con 0x7fbad0119040 2024-03-21T06:20:14.228 INFO:teuthology.orchestra.run.smithi184.stdout:2 2024-03-21T06:20:14.229 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.229+0000 7fbad6d5c640 1 -- 172.21.15.184:0/3350075716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad0119040 msgr2=0x7fbad01a7b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:14.229 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.229+0000 7fbad6d5c640 1 --2- 172.21.15.184:0/3350075716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad0119040 0x7fbad01a7b60 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:14.229 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.229+0000 7fbad6d5c640 1 -- 172.21.15.184:0/3350075716 shutdown_connections 2024-03-21T06:20:14.229 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.229+0000 7fbad6d5c640 1 --2- 172.21.15.184:0/3350075716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad0119040 0x7fbad01a7b60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:14.229 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.229+0000 7fbad6d5c640 1 -- 172.21.15.184:0/3350075716 >> 172.21.15.184:0/3350075716 conn(0x7fbad006d0e0 msgr2=0x7fbad0111da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:14.229 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.229+0000 7fbad6d5c640 1 -- 172.21.15.184:0/3350075716 shutdown_connections 2024-03-21T06:20:14.229 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.229+0000 7fbad6d5c640 1 -- 172.21.15.184:0/3350075716 wait complete. 2024-03-21T06:20:14.239 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 5945ad82-8437-4490-aca6-9c454678cd34 3 2024-03-21T06:20:14.288 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:14.287+0000 7fd10d7fdf00 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T06:20:14.346 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.340+0000 7f142842d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:14.346 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.345+0000 7f142842d640 1 Processor -- start 2024-03-21T06:20:14.346 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.346+0000 7f142842d640 1 -- start start 2024-03-21T06:20:14.347 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.346+0000 7f142842d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f142011bd20 0x7f142011e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:14.347 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.346+0000 7f142842d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f142011e710 con 0x7f1420119040 2024-03-21T06:20:14.348 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.346+0000 7f142842d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f142011f8b0 con 0x7f142011bd20 2024-03-21T06:20:14.348 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.346+0000 7f14259c8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f142011bd20 0x7f142011e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:14.348 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.346+0000 7f14261c9640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f1420119040 0x7f1420119450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.184:37604/0 (socket says 172.21.15.184:37604) 2024-03-21T06:20:14.348 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.346+0000 7f14259c8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f142011bd20 0x7f142011e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.184:41016/0 (socket says 172.21.15.184:41016) 2024-03-21T06:20:14.348 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.346+0000 7f14261c9640 1 -- 172.21.15.184:0/708569713 learned_addr learned my addr 172.21.15.184:0/708569713 (peer_addr_for_me v1:172.21.15.184:0/0) 2024-03-21T06:20:14.348 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.347+0000 7f14259c8640 1 -- 172.21.15.184:0/708569713 >> v1:172.21.15.96:6789/0 conn(0x7f1420119040 legacy=0x7f1420119450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:14.348 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.347+0000 7f14259c8640 1 -- 172.21.15.184:0/708569713 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142011fa20 con 0x7f142011bd20 2024-03-21T06:20:14.348 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.347+0000 7f14259c8640 1 --2- 172.21.15.184:0/708569713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f142011bd20 0x7f142011e140 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd010a454a91a9b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:14.348 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.347+0000 7f14251c7640 1 -- 172.21.15.184:0/708569713 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1410009ce0 con 0x7f142011bd20 2024-03-21T06:20:14.348 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.347+0000 7f14251c7640 1 -- 172.21.15.184:0/708569713 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1410009e40 con 0x7f142011bd20 2024-03-21T06:20:14.348 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.347+0000 7f14251c7640 1 -- 172.21.15.184:0/708569713 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f141000e360 con 0x7f142011bd20 2024-03-21T06:20:14.351 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.348+0000 7f142842d640 1 -- 172.21.15.184:0/708569713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f142011bd20 msgr2=0x7f142011e140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:14.351 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.348+0000 7f142842d640 1 --2- 172.21.15.184:0/708569713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f142011bd20 0x7f142011e140 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:14.351 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.348+0000 7f142842d640 1 -- 172.21.15.184:0/708569713 shutdown_connections 2024-03-21T06:20:14.352 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.348+0000 7f142842d640 1 --2- 172.21.15.184:0/708569713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f142011bd20 0x7f142011e140 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:14.352 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.348+0000 7f142842d640 1 -- 172.21.15.184:0/708569713 >> 172.21.15.184:0/708569713 conn(0x7f142006d0e0 msgr2=0x7f142006d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:14.352 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.348+0000 7f142842d640 1 -- 172.21.15.184:0/708569713 shutdown_connections 2024-03-21T06:20:14.352 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.348+0000 7f142842d640 1 -- 172.21.15.184:0/708569713 wait complete. 2024-03-21T06:20:14.352 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.348+0000 7f142842d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:14.352 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.349+0000 7f142842d640 1 Processor -- start 2024-03-21T06:20:14.352 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.349+0000 7f142842d640 1 -- start start 2024-03-21T06:20:14.352 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.349+0000 7f142842d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1420119040 0x7f14201a7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:14.352 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.349+0000 7f142842d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14201a8280 con 0x7f1420119040 2024-03-21T06:20:14.352 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.349+0000 7f14261c9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1420119040 0x7f14201a7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:14.352 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.349+0000 7f14261c9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1420119040 0x7f14201a7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.184:41030/0 (socket says 172.21.15.184:41030) 2024-03-21T06:20:14.352 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.349+0000 7f14261c9640 1 -- 172.21.15.184:0/3292955519 learned_addr learned my addr 172.21.15.184:0/3292955519 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-03-21T06:20:14.352 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.350+0000 7f14261c9640 1 -- 172.21.15.184:0/3292955519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14201a83c0 con 0x7f1420119040 2024-03-21T06:20:14.352 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.350+0000 7f14261c9640 1 --2- 172.21.15.184:0/3292955519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1420119040 0x7f14201a7cb0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:14.353 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.350+0000 7f1416ffd640 1 -- 172.21.15.184:0/3292955519 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f141c0059d0 con 0x7f1420119040 2024-03-21T06:20:14.353 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.350+0000 7f1416ffd640 1 -- 172.21.15.184:0/3292955519 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f141c005b30 con 0x7f1420119040 2024-03-21T06:20:14.353 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.350+0000 7f1416ffd640 1 -- 172.21.15.184:0/3292955519 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f141c005dd0 con 0x7f1420119040 2024-03-21T06:20:14.353 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.350+0000 7f142842d640 1 -- 172.21.15.184:0/3292955519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14201a8670 con 0x7f1420119040 2024-03-21T06:20:14.353 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.350+0000 7f142842d640 1 -- 172.21.15.184:0/3292955519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14201ac4a0 con 0x7f1420119040 2024-03-21T06:20:14.353 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.350+0000 7f1416ffd640 1 -- 172.21.15.184:0/3292955519 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f141c007440 con 0x7f1420119040 2024-03-21T06:20:14.353 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.350+0000 7f1416ffd640 1 -- 172.21.15.184:0/3292955519 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1092+0+0 (crc 0 0 0) 0x7f141c00f230 con 0x7f1420119040 2024-03-21T06:20:14.353 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.351+0000 7f142842d640 1 -- 172.21.15.184:0/3292955519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1420073c60 con 0x7f1420119040 2024-03-21T06:20:14.355 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.355+0000 7f1416ffd640 1 -- 172.21.15.184:0/3292955519 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (crc 0 0 0) 0x7f141c0076f0 con 0x7f1420119040 2024-03-21T06:20:14.419 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:14.419+0000 7fd10d7fdf00 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T06:20:14.427 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.427+0000 7f142842d640 1 -- 172.21.15.184:0/3292955519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5945ad82-8437-4490-aca6-9c454678cd34", "id": 3} v 0) v1 -- 0x7f142011bf30 con 0x7f1420119040 2024-03-21T06:20:14.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.431+0000 7f1416ffd640 1 -- 172.21.15.184:0/3292955519 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5945ad82-8437-4490-aca6-9c454678cd34", "id": 3}]=0 v7) v1 ==== 112+0+2 (crc 0 0 0) 0x7f141c00c840 con 0x7f1420119040 2024-03-21T06:20:14.431 INFO:teuthology.orchestra.run.smithi184.stdout:3 2024-03-21T06:20:14.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.432+0000 7f142842d640 1 -- 172.21.15.184:0/3292955519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1420119040 msgr2=0x7f14201a7cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:14.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.432+0000 7f142842d640 1 --2- 172.21.15.184:0/3292955519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1420119040 0x7f14201a7cb0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:14.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.432+0000 7f142842d640 1 -- 172.21.15.184:0/3292955519 shutdown_connections 2024-03-21T06:20:14.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.432+0000 7f142842d640 1 --2- 172.21.15.184:0/3292955519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1420119040 0x7f14201a7cb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:14.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.432+0000 7f142842d640 1 -- 172.21.15.184:0/3292955519 >> 172.21.15.184:0/3292955519 conn(0x7f142006d0e0 msgr2=0x7f1420111da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:14.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.432+0000 7f142842d640 1 -- 172.21.15.184:0/3292955519 shutdown_connections 2024-03-21T06:20:14.433 INFO:teuthology.orchestra.run.smithi184.stderr:2024-03-21T06:20:14.432+0000 7f142842d640 1 -- 172.21.15.184:0/3292955519 wait complete. 2024-03-21T06:20:14.442 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2024-03-21T06:20:14.480 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-21T06:20:14.480 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-03-21T06:20:14.527 INFO:tasks.ceph.osd.0:Started 2024-03-21T06:20:14.619 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2024-03-21T06:20:14.620 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:14.539+0000 7fd10d7fdf00 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T06:20:14.620 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:14.584+0000 7fd10d7fdf00 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T06:20:14.655 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-21T06:20:14.655 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-03-21T06:20:14.696 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:14.696+0000 7fd10d7fdf00 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T06:20:14.700 INFO:tasks.ceph.osd.1:Started 2024-03-21T06:20:14.700 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2024-03-21T06:20:14.734 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-21T06:20:14.734 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-03-21T06:20:14.778 INFO:tasks.ceph.osd.2:Started 2024-03-21T06:20:14.778 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2024-03-21T06:20:14.799 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:14.799+0000 7fd10d7fdf00 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T06:20:14.810 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-21T06:20:14.811 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-03-21T06:20:14.853 INFO:tasks.ceph.osd.3:Started 2024-03-21T06:20:14.854 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T06:20:14.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.969+0000 7fa502b9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:14.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.974+0000 7fa502b9c640 1 Processor -- start 2024-03-21T06:20:14.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.974+0000 7fa502b9c640 1 -- start start 2024-03-21T06:20:14.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.975+0000 7fa502b9c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa4fc09cda0 0x7fa4fc139450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:14.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.975+0000 7fa502b9c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4fc06e410 con 0x7fa4fc09b360 2024-03-21T06:20:14.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.975+0000 7fa502b9c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4fc09b770 con 0x7fa4fc09cda0 2024-03-21T06:20:14.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.975+0000 7fa4fbfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa4fc09cda0 0x7fa4fc139450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:14.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.975+0000 7fa500938640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fa4fc09b360 0x7fa4fc1315d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:60210/0 (socket says 172.21.15.96:60210) 2024-03-21T06:20:14.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.975+0000 7fa4fbfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa4fc09cda0 0x7fa4fc139450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37586/0 (socket says 172.21.15.96:37586) 2024-03-21T06:20:14.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.975+0000 7fa500938640 1 -- 172.21.15.96:0/1940938370 learned_addr learned my addr 172.21.15.96:0/1940938370 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:14.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.975+0000 7fa4fbfff640 1 -- 172.21.15.96:0/1940938370 >> v1:172.21.15.96:6789/0 conn(0x7fa4fc09b360 legacy=0x7fa4fc1315d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:14.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.975+0000 7fa4fbfff640 1 -- 172.21.15.96:0/1940938370 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc09d1b0 con 0x7fa4fc09cda0 2024-03-21T06:20:14.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.975+0000 7fa4fbfff640 1 --2- 172.21.15.96:0/1940938370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4fc09cda0 0x7fa4fc139450 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408e13dedd3de2a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:14.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.975+0000 7fa4fb7fe640 1 -- 172.21.15.96:0/1940938370 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa4e8009ce0 con 0x7fa4fc09cda0 2024-03-21T06:20:14.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.976+0000 7fa4fb7fe640 1 -- 172.21.15.96:0/1940938370 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa4e8009e40 con 0x7fa4fc09cda0 2024-03-21T06:20:14.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.976+0000 7fa4fb7fe640 1 -- 172.21.15.96:0/1940938370 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa4e800e360 con 0x7fa4fc09cda0 2024-03-21T06:20:14.977 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:14.975+0000 7fd10d7fdf00 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T06:20:14.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.976+0000 7fa502b9c640 1 -- 172.21.15.96:0/1940938370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4fc09cda0 msgr2=0x7fa4fc139450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:14.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.976+0000 7fa502b9c640 1 --2- 172.21.15.96:0/1940938370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4fc09cda0 0x7fa4fc139450 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:14.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.976+0000 7fa502b9c640 1 -- 172.21.15.96:0/1940938370 shutdown_connections 2024-03-21T06:20:14.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.976+0000 7fa502b9c640 1 --2- 172.21.15.96:0/1940938370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4fc09cda0 0x7fa4fc139450 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:14.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.976+0000 7fa502b9c640 1 -- 172.21.15.96:0/1940938370 >> 172.21.15.96:0/1940938370 conn(0x7fa4fc0a42c0 msgr2=0x7fa4fc0a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:14.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.976+0000 7fa502b9c640 1 -- 172.21.15.96:0/1940938370 shutdown_connections 2024-03-21T06:20:14.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.977+0000 7fa502b9c640 1 -- 172.21.15.96:0/1940938370 wait complete. 2024-03-21T06:20:14.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.977+0000 7fa502b9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:14.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.978+0000 7fa502b9c640 1 Processor -- start 2024-03-21T06:20:14.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.978+0000 7fa502b9c640 1 -- start start 2024-03-21T06:20:14.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.979+0000 7fa502b9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4fc09b360 0x7fa4fc1c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:14.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.979+0000 7fa502b9c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc11e420 con 0x7fa4fc09b360 2024-03-21T06:20:14.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.979+0000 7fa500938640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4fc09b360 0x7fa4fc1c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:14.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.979+0000 7fa500938640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4fc09b360 0x7fa4fc1c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37594/0 (socket says 172.21.15.96:37594) 2024-03-21T06:20:14.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.979+0000 7fa500938640 1 -- 172.21.15.96:0/4108870606 learned_addr learned my addr 172.21.15.96:0/4108870606 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:14.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.979+0000 7fa500938640 1 -- 172.21.15.96:0/4108870606 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc073030 con 0x7fa4fc09b360 2024-03-21T06:20:14.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.979+0000 7fa500938640 1 --2- 172.21.15.96:0/4108870606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4fc09b360 0x7fa4fc1c8d20 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:14.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.979+0000 7fa4f97fa640 1 -- 172.21.15.96:0/4108870606 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa4f00059d0 con 0x7fa4fc09b360 2024-03-21T06:20:14.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.979+0000 7fa4f97fa640 1 -- 172.21.15.96:0/4108870606 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa4f0005b30 con 0x7fa4fc09b360 2024-03-21T06:20:14.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.979+0000 7fa502b9c640 1 -- 172.21.15.96:0/4108870606 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4fc11cd30 con 0x7fa4fc09b360 2024-03-21T06:20:14.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.979+0000 7fa4f97fa640 1 -- 172.21.15.96:0/4108870606 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa4f0005dd0 con 0x7fa4fc09b360 2024-03-21T06:20:14.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.979+0000 7fa502b9c640 1 -- 172.21.15.96:0/4108870606 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4fc11d510 con 0x7fa4fc09b360 2024-03-21T06:20:14.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.979+0000 7fa4f97fa640 1 -- 172.21.15.96:0/4108870606 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7fa4f0007440 con 0x7fa4fc09b360 2024-03-21T06:20:14.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.980+0000 7fa4f97fa640 1 -- 172.21.15.96:0/4108870606 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1207+0+0 (crc 0 0 0) 0x7fa4f000cc50 con 0x7fa4fc09b360 2024-03-21T06:20:14.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.981+0000 7fa502b9c640 1 -- 172.21.15.96:0/4108870606 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4fc11cf20 con 0x7fa4fc09b360 2024-03-21T06:20:14.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:14.983+0000 7fa4f97fa640 1 -- 172.21.15.96:0/4108870606 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (crc 0 0 0) 0x7fa4fc11cf20 con 0x7fa4fc09b360 2024-03-21T06:20:15.027 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:15.027+0000 7fd10d7fdf00 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T06:20:15.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:15.045+0000 7fa502b9c640 1 -- 172.21.15.96:0/4108870606 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4fc1392c0 con 0x7fa4fc09b360 2024-03-21T06:20:15.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:15.046+0000 7fa4f97fa640 1 -- 172.21.15.96:0/4108870606 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) v1 ==== 74+0+4152 (crc 0 0 0) 0x7fa4fc073030 con 0x7fa4fc09b360 2024-03-21T06:20:15.047 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:20:15.047 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":7,"fsid":"e880df84-2d7c-4386-b5e9-f3879b9ab9d5","created":"2024-03-21T06:20:12.270291+0000","modified":"2024-03-21T06:20:14.428643+0000","last_up_change":"0.000000","last_in_change":"2024-03-21T06:20:14.428643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"ffab25be-1f6f-450d-a8b4-da7f86f3eaeb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3266cd80-6286-4a69-933a-00cd48d67487","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5a8b1e5a-d4ed-4c96-b301-4173028235d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5945ad82-8437-4490-aca6-9c454678cd34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T06:20:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:15.047+0000 7fa502b9c640 1 -- 172.21.15.96:0/4108870606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4fc09b360 msgr2=0x7fa4fc1c8d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:15.047+0000 7fa502b9c640 1 --2- 172.21.15.96:0/4108870606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4fc09b360 0x7fa4fc1c8d20 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:15.047+0000 7fa502b9c640 1 -- 172.21.15.96:0/4108870606 shutdown_connections 2024-03-21T06:20:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:15.047+0000 7fa502b9c640 1 --2- 172.21.15.96:0/4108870606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4fc09b360 0x7fa4fc1c8d20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:15.047+0000 7fa502b9c640 1 -- 172.21.15.96:0/4108870606 >> 172.21.15.96:0/4108870606 conn(0x7fa4fc0a42c0 msgr2=0x7fa4fc0a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:15.047+0000 7fa502b9c640 1 -- 172.21.15.96:0/4108870606 shutdown_connections 2024-03-21T06:20:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:15.047+0000 7fa502b9c640 1 -- 172.21.15.96:0/4108870606 wait complete. 2024-03-21T06:20:15.059 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-21T06:20:15.059 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-21T06:20:15.212 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:15.211+0000 7fd10d7fdf00 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T06:20:15.462 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-21T06:20:15.561 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.557+0000 7f9465d1c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:15.562 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.562+0000 7f9465d1c640 1 Processor -- start 2024-03-21T06:20:15.562 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.562+0000 7f9465d1c640 1 -- start start 2024-03-21T06:20:15.563 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.562+0000 7f9465d1c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9460134640 0x7f9460134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:15.564 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.562+0000 7f9465d1c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946009f320 con 0x7f946009ba50 2024-03-21T06:20:15.564 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.562+0000 7f9465d1c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946009d200 con 0x7f9460134640 2024-03-21T06:20:15.564 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.563+0000 7f945f7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9460134640 0x7f9460134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:15.564 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.563+0000 7f945effd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f946009ba50 0x7f946009be60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:60222/0 (socket says 172.21.15.96:60222) 2024-03-21T06:20:15.564 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.563+0000 7f945effd640 1 -- 172.21.15.96:0/1023658926 learned_addr learned my addr 172.21.15.96:0/1023658926 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:15.564 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.563+0000 7f945f7fe640 1 -- 172.21.15.96:0/1023658926 >> v1:172.21.15.96:6789/0 conn(0x7f946009ba50 legacy=0x7f946009be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:15.564 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.563+0000 7f945f7fe640 1 -- 172.21.15.96:0/1023658926 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94600a3d70 con 0x7f9460134640 2024-03-21T06:20:15.564 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.563+0000 7f945f7fe640 1 --2- 172.21.15.96:0/1023658926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9460134640 0x7f9460134a50 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7fda1d9394e42f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:15.564 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.563+0000 7f945e7fc640 1 -- 172.21.15.96:0/1023658926 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9450009ce0 con 0x7f9460134640 2024-03-21T06:20:15.564 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.563+0000 7f945e7fc640 1 -- 172.21.15.96:0/1023658926 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f9450009e40 con 0x7f9460134640 2024-03-21T06:20:15.564 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.563+0000 7f945e7fc640 1 -- 172.21.15.96:0/1023658926 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f945000e360 con 0x7f9460134640 2024-03-21T06:20:15.566 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.564+0000 7f9465d1c640 1 -- 172.21.15.96:0/1023658926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9460134640 msgr2=0x7f9460134a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:15.566 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.564+0000 7f9465d1c640 1 --2- 172.21.15.96:0/1023658926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9460134640 0x7f9460134a50 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:15.566 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.564+0000 7f9465d1c640 1 -- 172.21.15.96:0/1023658926 shutdown_connections 2024-03-21T06:20:15.566 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.564+0000 7f9465d1c640 1 --2- 172.21.15.96:0/1023658926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9460134640 0x7f9460134a50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:15.566 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.564+0000 7f9465d1c640 1 -- 172.21.15.96:0/1023658926 >> 172.21.15.96:0/1023658926 conn(0x7f946006c4b0 msgr2=0x7f946006c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:15.566 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.564+0000 7f9465d1c640 1 -- 172.21.15.96:0/1023658926 shutdown_connections 2024-03-21T06:20:15.566 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.564+0000 7f9465d1c640 1 -- 172.21.15.96:0/1023658926 wait complete. 2024-03-21T06:20:15.566 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.564+0000 7f9465d1c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:15.566 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.565+0000 7f9465d1c640 1 Processor -- start 2024-03-21T06:20:15.567 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.565+0000 7f9465d1c640 1 -- start start 2024-03-21T06:20:15.568 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.566+0000 7f9465d1c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f946009ba50 0x7f946011e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:15.569 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.566+0000 7f9465d1c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946011ff90 con 0x7f946009ba50 2024-03-21T06:20:15.569 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.566+0000 7f945f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f946009ba50 0x7f946011e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:15.569 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.566+0000 7f945f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f946009ba50 0x7f946011e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37602/0 (socket says 172.21.15.96:37602) 2024-03-21T06:20:15.569 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.566+0000 7f945f7fe640 1 -- 172.21.15.96:0/3812175373 learned_addr learned my addr 172.21.15.96:0/3812175373 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:15.569 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.566+0000 7f945f7fe640 1 -- 172.21.15.96:0/3812175373 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946011de50 con 0x7f946009ba50 2024-03-21T06:20:15.569 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.567+0000 7f945f7fe640 1 --2- 172.21.15.96:0/3812175373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f946009ba50 0x7f946011e210 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:15.569 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.567+0000 7f9447fff640 1 -- 172.21.15.96:0/3812175373 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9450014620 con 0x7f946009ba50 2024-03-21T06:20:15.569 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.567+0000 7f9447fff640 1 -- 172.21.15.96:0/3812175373 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f9450014780 con 0x7f946009ba50 2024-03-21T06:20:15.569 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.567+0000 7f9465d1c640 1 -- 172.21.15.96:0/3812175373 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94601201c0 con 0x7f946009ba50 2024-03-21T06:20:15.569 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.567+0000 7f9447fff640 1 -- 172.21.15.96:0/3812175373 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9450014a20 con 0x7f946009ba50 2024-03-21T06:20:15.569 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.567+0000 7f9465d1c640 1 -- 172.21.15.96:0/3812175373 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94601215f0 con 0x7f946009ba50 2024-03-21T06:20:15.569 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.567+0000 7f9447fff640 1 -- 172.21.15.96:0/3812175373 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (crc 0 0 0) 0x7f9450011380 con 0x7f946009ba50 2024-03-21T06:20:15.570 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.567+0000 7f9447fff640 1 -- 172.21.15.96:0/3812175373 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1207+0+0 (crc 0 0 0) 0x7f9450019040 con 0x7f946009ba50 2024-03-21T06:20:15.570 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.568+0000 7f9465d1c640 1 -- 172.21.15.96:0/3812175373 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946011fd00 con 0x7f946009ba50 2024-03-21T06:20:15.570 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.570+0000 7f9447fff640 1 -- 172.21.15.96:0/3812175373 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (crc 0 0 0) 0x7f9450010bf0 con 0x7f946009ba50 2024-03-21T06:20:15.632 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.632+0000 7f9465d1c640 1 -- 172.21.15.96:0/3812175373 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f946011dbf0 con 0x7f946009ba50 2024-03-21T06:20:15.633 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.632+0000 7f9447fff640 1 -- 172.21.15.96:0/3812175373 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) v1 ==== 74+0+4152 (crc 0 0 0) 0x7f9450010e40 con 0x7f946009ba50 2024-03-21T06:20:15.634 INFO:teuthology.misc.health.smithi096.stdout: 2024-03-21T06:20:15.634 INFO:teuthology.misc.health.smithi096.stdout:{"epoch":7,"fsid":"e880df84-2d7c-4386-b5e9-f3879b9ab9d5","created":"2024-03-21T06:20:12.270291+0000","modified":"2024-03-21T06:20:14.428643+0000","last_up_change":"0.000000","last_in_change":"2024-03-21T06:20:14.428643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"ffab25be-1f6f-450d-a8b4-da7f86f3eaeb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3266cd80-6286-4a69-933a-00cd48d67487","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5a8b1e5a-d4ed-4c96-b301-4173028235d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5945ad82-8437-4490-aca6-9c454678cd34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T06:20:15.634 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.633+0000 7f9465d1c640 1 -- 172.21.15.96:0/3812175373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f946009ba50 msgr2=0x7f946011e210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:15.635 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.633+0000 7f9465d1c640 1 --2- 172.21.15.96:0/3812175373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f946009ba50 0x7f946011e210 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:15.635 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.633+0000 7f9465d1c640 1 -- 172.21.15.96:0/3812175373 shutdown_connections 2024-03-21T06:20:15.635 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.633+0000 7f9465d1c640 1 --2- 172.21.15.96:0/3812175373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f946009ba50 0x7f946011e210 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:15.635 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.633+0000 7f9465d1c640 1 -- 172.21.15.96:0/3812175373 >> 172.21.15.96:0/3812175373 conn(0x7f946006c4b0 msgr2=0x7f94601bc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:15.635 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.633+0000 7f9465d1c640 1 -- 172.21.15.96:0/3812175373 shutdown_connections 2024-03-21T06:20:15.635 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:15.633+0000 7f9465d1c640 1 -- 172.21.15.96:0/3812175373 wait complete. 2024-03-21T06:20:15.644 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-03-21T06:20:16.118 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:16.118+0000 7fd10d7fdf00 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T06:20:16.410 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:16.410+0000 7fd10d7fdf00 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T06:20:16.465 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:16.465+0000 7fd10d7fdf00 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T06:20:16.592 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:16.592+0000 7fd10d7fdf00 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T06:20:17.612 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:20:17.611+0000 7fd0ae0b4640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-03-21T06:20:22.048 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-21T06:20:22.148 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.144+0000 7f332626d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:22.148 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.148+0000 7f332626d640 1 Processor -- start 2024-03-21T06:20:22.149 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.148+0000 7f332626d640 1 -- start start 2024-03-21T06:20:22.150 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.149+0000 7f332626d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3320134a50 0x7f332009c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:22.150 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.149+0000 7f332626d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332009f320 con 0x7f3320134640 2024-03-21T06:20:22.150 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.149+0000 7f332626d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f332009d200 con 0x7f3320134a50 2024-03-21T06:20:22.150 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.149+0000 7f331f7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3320134a50 0x7f332009c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:22.151 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.149+0000 7f331ffff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f3320134640 0x7f332009ba50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33038/0 (socket says 172.21.15.96:33038) 2024-03-21T06:20:22.151 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.149+0000 7f331ffff640 1 -- 172.21.15.96:0/1718739695 learned_addr learned my addr 172.21.15.96:0/1718739695 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:22.151 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.150+0000 7f331f7fe640 1 -- 172.21.15.96:0/1718739695 >> v1:172.21.15.96:6789/0 conn(0x7f3320134640 legacy=0x7f332009ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:22.151 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.150+0000 7f331f7fe640 1 -- 172.21.15.96:0/1718739695 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33200a3d70 con 0x7f3320134a50 2024-03-21T06:20:22.151 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.150+0000 7f331f7fe640 1 --2- 172.21.15.96:0/1718739695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3320134a50 0x7f332009c160 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5c002f92b587e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:22.151 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.150+0000 7f331effd640 1 -- 172.21.15.96:0/1718739695 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3310009ce0 con 0x7f3320134a50 2024-03-21T06:20:22.151 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.150+0000 7f331effd640 1 -- 172.21.15.96:0/1718739695 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3310009e40 con 0x7f3320134a50 2024-03-21T06:20:22.151 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.150+0000 7f331effd640 1 -- 172.21.15.96:0/1718739695 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f331000e360 con 0x7f3320134a50 2024-03-21T06:20:22.152 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.151+0000 7f332626d640 1 -- 172.21.15.96:0/1718739695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3320134a50 msgr2=0x7f332009c160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:22.152 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.151+0000 7f332626d640 1 --2- 172.21.15.96:0/1718739695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3320134a50 0x7f332009c160 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:22.152 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.151+0000 7f332626d640 1 -- 172.21.15.96:0/1718739695 shutdown_connections 2024-03-21T06:20:22.152 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.151+0000 7f332626d640 1 --2- 172.21.15.96:0/1718739695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3320134a50 0x7f332009c160 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:22.152 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.151+0000 7f332626d640 1 -- 172.21.15.96:0/1718739695 >> 172.21.15.96:0/1718739695 conn(0x7f332006c4b0 msgr2=0x7f332006c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:22.153 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.151+0000 7f332626d640 1 -- 172.21.15.96:0/1718739695 shutdown_connections 2024-03-21T06:20:22.153 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.151+0000 7f332626d640 1 -- 172.21.15.96:0/1718739695 wait complete. 2024-03-21T06:20:22.153 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.151+0000 7f332626d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:22.153 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.152+0000 7f332626d640 1 Processor -- start 2024-03-21T06:20:22.153 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.152+0000 7f332626d640 1 -- start start 2024-03-21T06:20:22.155 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.153+0000 7f332626d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3320134640 0x7f332011d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:22.155 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.153+0000 7f332626d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3320121920 con 0x7f3320134640 2024-03-21T06:20:22.155 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.153+0000 7f331ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3320134640 0x7f332011d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:22.155 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.153+0000 7f331ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3320134640 0x7f332011d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35020/0 (socket says 172.21.15.96:35020) 2024-03-21T06:20:22.155 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.153+0000 7f331ffff640 1 -- 172.21.15.96:0/1451244343 learned_addr learned my addr 172.21.15.96:0/1451244343 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:22.155 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.153+0000 7f331ffff640 1 -- 172.21.15.96:0/1451244343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33200678d0 con 0x7f3320134640 2024-03-21T06:20:22.155 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.153+0000 7f331ffff640 1 --2- 172.21.15.96:0/1451244343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3320134640 0x7f332011d7a0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:22.155 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.153+0000 7f331cff9640 1 -- 172.21.15.96:0/1451244343 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f33140059d0 con 0x7f3320134640 2024-03-21T06:20:22.155 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.153+0000 7f331cff9640 1 -- 172.21.15.96:0/1451244343 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3314005b30 con 0x7f3320134640 2024-03-21T06:20:22.155 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.153+0000 7f332626d640 1 -- 172.21.15.96:0/1451244343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332011f3c0 con 0x7f3320134640 2024-03-21T06:20:22.155 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.153+0000 7f331cff9640 1 -- 172.21.15.96:0/1451244343 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3314005dd0 con 0x7f3320134640 2024-03-21T06:20:22.155 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.153+0000 7f332626d640 1 -- 172.21.15.96:0/1451244343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3320120320 con 0x7f3320134640 2024-03-21T06:20:22.156 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.154+0000 7f331cff9640 1 -- 172.21.15.96:0/1451244343 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) v1 ==== 45326+0+0 (crc 0 0 0) 0x7f33140070b0 con 0x7f3320134640 2024-03-21T06:20:22.156 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.154+0000 7f331cff9640 1 --2- 172.21.15.96:0/1451244343 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f330c0381d0 0x7f330c03a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:22.156 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.154+0000 7f332626d640 1 -- 172.21.15.96:0/1451244343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3320062f10 con 0x7f3320134640 2024-03-21T06:20:22.156 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.154+0000 7f331cff9640 1 -- 172.21.15.96:0/1451244343 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2319+0+0 (crc 0 0 0) 0x7f33140473f0 con 0x7f3320134640 2024-03-21T06:20:22.156 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.154+0000 7f331f7fe640 1 --2- 172.21.15.96:0/1451244343 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f330c0381d0 0x7f330c03a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:22.156 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.155+0000 7f331f7fe640 1 --2- 172.21.15.96:0/1451244343 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f330c0381d0 0x7f330c03a6e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:22.160 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.160+0000 7f331cff9640 1 -- 172.21.15.96:0/1451244343 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f331400da20 con 0x7f3320134640 2024-03-21T06:20:22.345 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.345+0000 7f332626d640 1 -- 172.21.15.96:0/1451244343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33200678d0 con 0x7f3320134640 2024-03-21T06:20:22.346 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.345+0000 7f331cff9640 1 -- 172.21.15.96:0/1451244343 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+6907 (crc 0 0 0) 0x7f33200678d0 con 0x7f3320134640 2024-03-21T06:20:22.346 INFO:teuthology.misc.health.smithi096.stdout: 2024-03-21T06:20:22.346 INFO:teuthology.misc.health.smithi096.stdout:{"epoch":11,"fsid":"e880df84-2d7c-4386-b5e9-f3879b9ab9d5","created":"2024-03-21T06:20:12.270291+0000","modified":"2024-03-21T06:20:21.663803+0000","last_up_change":"2024-03-21T06:20:18.657108+0000","last_in_change":"2024-03-21T06:20:14.428643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T06:20:20.651568+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffab25be-1f6f-450d-a8b4-da7f86f3eaeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6801","nonce":932229279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":932229279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":932229279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6803","nonce":932229279}]},"public_addr":"172.21.15.96:6801/932229279","cluster_addr":"172.21.15.96:6802/932229279","heartbeat_back_addr":"172.21.15.96:6808/932229279","heartbeat_front_addr":"172.21.15.96:6803/932229279","state":["exists","up"]},{"osd":1,"uuid":"3266cd80-6286-4a69-933a-00cd48d67487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6805","nonce":13988474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":13988474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":13988474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6807","nonce":13988474}]},"public_addr":"172.21.15.96:6805/13988474","cluster_addr":"172.21.15.96:6804/13988474","heartbeat_back_addr":"172.21.15.96:6806/13988474","heartbeat_front_addr":"172.21.15.96:6807/13988474","state":["exists","up"]},{"osd":2,"uuid":"5a8b1e5a-d4ed-4c96-b301-4173028235d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":155456265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":155456265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":155456265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6815","nonce":155456265}]},"public_addr":"172.21.15.184:6804/155456265","cluster_addr":"172.21.15.184:6805/155456265","heartbeat_back_addr":"172.21.15.184:6814/155456265","heartbeat_front_addr":"172.21.15.184:6815/155456265","state":["exists","up"]},{"osd":3,"uuid":"5945ad82-8437-4490-aca6-9c454678cd34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":276480837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":276480837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":276480837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":276480837}]},"public_addr":"172.21.15.184:6811/276480837","cluster_addr":"172.21.15.184:6810/276480837","heartbeat_back_addr":"172.21.15.184:6808/276480837","heartbeat_front_addr":"172.21.15.184:6809/276480837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T06:20:22.349 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.348+0000 7f332626d640 1 -- 172.21.15.96:0/1451244343 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f330c0381d0 msgr2=0x7f330c03a6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:22.349 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.348+0000 7f332626d640 1 --2- 172.21.15.96:0/1451244343 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f330c0381d0 0x7f330c03a6e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:22.349 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.348+0000 7f332626d640 1 -- 172.21.15.96:0/1451244343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3320134640 msgr2=0x7f332011d7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:22.349 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.348+0000 7f332626d640 1 --2- 172.21.15.96:0/1451244343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3320134640 0x7f332011d7a0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:22.349 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.348+0000 7f332626d640 1 -- 172.21.15.96:0/1451244343 shutdown_connections 2024-03-21T06:20:22.349 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.348+0000 7f332626d640 1 --2- 172.21.15.96:0/1451244343 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f330c0381d0 0x7f330c03a6e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:22.349 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.348+0000 7f332626d640 1 --2- 172.21.15.96:0/1451244343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3320134640 0x7f332011d7a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:22.349 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.348+0000 7f332626d640 1 -- 172.21.15.96:0/1451244343 >> 172.21.15.96:0/1451244343 conn(0x7f332006c4b0 msgr2=0x7f33200a4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:22.349 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.348+0000 7f332626d640 1 -- 172.21.15.96:0/1451244343 shutdown_connections 2024-03-21T06:20:22.349 INFO:teuthology.misc.health.smithi096.stderr:2024-03-21T06:20:22.348+0000 7f332626d640 1 -- 172.21.15.96:0/1451244343 wait complete. 2024-03-21T06:20:22.360 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-03-21T06:20:22.360 INFO:tasks.ceph:Creating RBD pool 2024-03-21T06:20:22.360 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd pool create rbd 8 2024-03-21T06:20:22.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.464+0000 7f9577b1c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:22.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.470+0000 7f9577b1c640 1 Processor -- start 2024-03-21T06:20:22.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.471+0000 7f9577b1c640 1 -- start start 2024-03-21T06:20:22.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.471+0000 7f9577b1c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f957009b170 0x7f95701324f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:22.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.471+0000 7f9577b1c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9570099340 con 0x7f9570132a30 2024-03-21T06:20:22.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.471+0000 7f9577b1c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957009b650 con 0x7f957009b170 2024-03-21T06:20:22.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.472+0000 7f95758b8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f957009b170 0x7f95701324f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:22.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.472+0000 7f95750b7640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f9570132a30 0x7f9570099880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33054/0 (socket says 172.21.15.96:33054) 2024-03-21T06:20:22.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.472+0000 7f95750b7640 1 -- 172.21.15.96:0/4015098228 learned_addr learned my addr 172.21.15.96:0/4015098228 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:22.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.472+0000 7f95758b8640 1 -- 172.21.15.96:0/4015098228 >> v1:172.21.15.96:6789/0 conn(0x7f9570132a30 legacy=0x7f9570099880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:22.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.472+0000 7f95758b8640 1 -- 172.21.15.96:0/4015098228 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957009a9a0 con 0x7f957009b170 2024-03-21T06:20:22.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.472+0000 7f95758b8640 1 --2- 172.21.15.96:0/4015098228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f957009b170 0x7f95701324f0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a5d8e75ff0ea57 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:22.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.472+0000 7f95748b6640 1 -- 172.21.15.96:0/4015098228 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f956c009ce0 con 0x7f957009b170 2024-03-21T06:20:22.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.472+0000 7f95748b6640 1 -- 172.21.15.96:0/4015098228 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f956c009e40 con 0x7f957009b170 2024-03-21T06:20:22.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.472+0000 7f95748b6640 1 -- 172.21.15.96:0/4015098228 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f956c00e360 con 0x7f957009b170 2024-03-21T06:20:22.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.473+0000 7f9577b1c640 1 -- 172.21.15.96:0/4015098228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f957009b170 msgr2=0x7f95701324f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:22.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.473+0000 7f9577b1c640 1 --2- 172.21.15.96:0/4015098228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f957009b170 0x7f95701324f0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:22.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.473+0000 7f9577b1c640 1 -- 172.21.15.96:0/4015098228 shutdown_connections 2024-03-21T06:20:22.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.473+0000 7f9577b1c640 1 --2- 172.21.15.96:0/4015098228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f957009b170 0x7f95701324f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:22.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.473+0000 7f9577b1c640 1 -- 172.21.15.96:0/4015098228 >> 172.21.15.96:0/4015098228 conn(0x7f95700a0280 msgr2=0x7f95700a0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:22.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.473+0000 7f9577b1c640 1 -- 172.21.15.96:0/4015098228 shutdown_connections 2024-03-21T06:20:22.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.473+0000 7f9577b1c640 1 -- 172.21.15.96:0/4015098228 wait complete. 2024-03-21T06:20:22.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.473+0000 7f9577b1c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:22.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.474+0000 7f9577b1c640 1 Processor -- start 2024-03-21T06:20:22.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.474+0000 7f9577b1c640 1 -- start start 2024-03-21T06:20:22.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.475+0000 7f9577b1c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f957009b170 0x7f957006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:22.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.475+0000 7f9577b1c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957012fef0 con 0x7f957009b170 2024-03-21T06:20:22.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.475+0000 7f95758b8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f957009b170 0x7f957006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:22.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.475+0000 7f95758b8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f957009b170 0x7f957006da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35044/0 (socket says 172.21.15.96:35044) 2024-03-21T06:20:22.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.475+0000 7f95758b8640 1 -- 172.21.15.96:0/1934857937 learned_addr learned my addr 172.21.15.96:0/1934857937 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:22.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.475+0000 7f95758b8640 1 -- 172.21.15.96:0/1934857937 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9570131860 con 0x7f957009b170 2024-03-21T06:20:22.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.475+0000 7f95758b8640 1 --2- 172.21.15.96:0/1934857937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f957009b170 0x7f957006da10 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:22.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.475+0000 7f95667fc640 1 -- 172.21.15.96:0/1934857937 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f956c014410 con 0x7f957009b170 2024-03-21T06:20:22.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.475+0000 7f95667fc640 1 -- 172.21.15.96:0/1934857937 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f956c014570 con 0x7f957009b170 2024-03-21T06:20:22.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.475+0000 7f9577b1c640 1 -- 172.21.15.96:0/1934857937 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9570130170 con 0x7f957009b170 2024-03-21T06:20:22.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.475+0000 7f95667fc640 1 -- 172.21.15.96:0/1934857937 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f956c014810 con 0x7f957009b170 2024-03-21T06:20:22.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.475+0000 7f9577b1c640 1 -- 172.21.15.96:0/1934857937 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957006bd00 con 0x7f957009b170 2024-03-21T06:20:22.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.476+0000 7f95667fc640 1 -- 172.21.15.96:0/1934857937 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) v1 ==== 45326+0+0 (crc 0 0 0) 0x7f956c0149c0 con 0x7f957009b170 2024-03-21T06:20:22.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.476+0000 7f9577b1c640 1 -- 172.21.15.96:0/1934857937 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9570062f10 con 0x7f957009b170 2024-03-21T06:20:22.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.476+0000 7f95667fc640 1 --2- 172.21.15.96:0/1934857937 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f95540381d0 0x7f955403a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:22.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.476+0000 7f95667fc640 1 -- 172.21.15.96:0/1934857937 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2319+0+0 (crc 0 0 0) 0x7f956c019040 con 0x7f957009b170 2024-03-21T06:20:22.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.477+0000 7f95750b7640 1 --2- 172.21.15.96:0/1934857937 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f95540381d0 0x7f955403a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:22.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.477+0000 7f95750b7640 1 --2- 172.21.15.96:0/1934857937 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f95540381d0 0x7f955403a6e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:22.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.481+0000 7f95667fc640 1 -- 172.21.15.96:0/1934857937 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f956c00f990 con 0x7f957009b170 2024-03-21T06:20:22.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.655+0000 7f9577b1c640 1 -- 172.21.15.96:0/1934857937 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f957006b2e0 con 0x7f957009b170 2024-03-21T06:20:22.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.673+0000 7f95667fc640 1 -- 172.21.15.96:0/1934857937 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v12) v1 ==== 109+0+0 (crc 0 0 0) 0x7f956c00f7b0 con 0x7f957009b170 2024-03-21T06:20:22.674 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'rbd' created 2024-03-21T06:20:22.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.676+0000 7f9577b1c640 1 -- 172.21.15.96:0/1934857937 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f95540381d0 msgr2=0x7f955403a6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:22.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.676+0000 7f9577b1c640 1 --2- 172.21.15.96:0/1934857937 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f95540381d0 0x7f955403a6e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:22.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.676+0000 7f9577b1c640 1 -- 172.21.15.96:0/1934857937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f957009b170 msgr2=0x7f957006da10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:22.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.676+0000 7f9577b1c640 1 --2- 172.21.15.96:0/1934857937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f957009b170 0x7f957006da10 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:22.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.676+0000 7f9577b1c640 1 -- 172.21.15.96:0/1934857937 shutdown_connections 2024-03-21T06:20:22.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.676+0000 7f9577b1c640 1 --2- 172.21.15.96:0/1934857937 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f95540381d0 0x7f955403a6e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:22.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.676+0000 7f9577b1c640 1 --2- 172.21.15.96:0/1934857937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f957009b170 0x7f957006da10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:22.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.676+0000 7f9577b1c640 1 -- 172.21.15.96:0/1934857937 >> 172.21.15.96:0/1934857937 conn(0x7f95700a0280 msgr2=0x7f95701ba2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:22.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.676+0000 7f9577b1c640 1 -- 172.21.15.96:0/1934857937 shutdown_connections 2024-03-21T06:20:22.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.676+0000 7f9577b1c640 1 -- 172.21.15.96:0/1934857937 wait complete. 2024-03-21T06:20:22.692 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-03-21T06:20:22.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.794+0000 7f99f675c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:22.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.804+0000 7f99f675c640 1 Processor -- start 2024-03-21T06:20:22.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.804+0000 7f99f675c640 1 -- start start 2024-03-21T06:20:22.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.805+0000 7f99f675c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f99f0099f20 0x7f99f009a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:22.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.805+0000 7f99f675c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f009b380 con 0x7f99f0130540 2024-03-21T06:20:22.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.805+0000 7f99f675c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f009a870 con 0x7f99f0099f20 2024-03-21T06:20:22.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.805+0000 7f99effff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f99f0099f20 0x7f99f009a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:22.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.805+0000 7f99ef7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f99f0130540 0x7f99f0130950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33056/0 (socket says 172.21.15.96:33056) 2024-03-21T06:20:22.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.805+0000 7f99ef7fe640 1 -- 172.21.15.96:0/3896202349 learned_addr learned my addr 172.21.15.96:0/3896202349 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:22.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.805+0000 7f99effff640 1 -- 172.21.15.96:0/3896202349 >> v1:172.21.15.96:6789/0 conn(0x7f99f0130540 legacy=0x7f99f0130950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:22.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.805+0000 7f99effff640 1 -- 172.21.15.96:0/3896202349 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f0099490 con 0x7f99f0099f20 2024-03-21T06:20:22.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.805+0000 7f99eeffd640 1 -- 172.21.15.96:0/3896202349 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052269710 0 0) 0x7f99f009b380 con 0x7f99f0130540 2024-03-21T06:20:22.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.805+0000 7f99effff640 1 --2- 172.21.15.96:0/3896202349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99f0099f20 0x7f99f009a330 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1856132e94b2b18e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:22.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.805+0000 7f99eeffd640 1 -- 172.21.15.96:0/3896202349 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f99e0009ce0 con 0x7f99f0099f20 2024-03-21T06:20:22.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.805+0000 7f99eeffd640 1 -- 172.21.15.96:0/3896202349 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f99e0009e40 con 0x7f99f0099f20 2024-03-21T06:20:22.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.805+0000 7f99eeffd640 1 -- 172.21.15.96:0/3896202349 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f99e000e360 con 0x7f99f0099f20 2024-03-21T06:20:22.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.806+0000 7f99f675c640 1 -- 172.21.15.96:0/3896202349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99f0099f20 msgr2=0x7f99f009a330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:22.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.806+0000 7f99f675c640 1 --2- 172.21.15.96:0/3896202349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99f0099f20 0x7f99f009a330 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:22.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.806+0000 7f99f675c640 1 -- 172.21.15.96:0/3896202349 shutdown_connections 2024-03-21T06:20:22.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.806+0000 7f99f675c640 1 --2- 172.21.15.96:0/3896202349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99f0099f20 0x7f99f009a330 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:22.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.806+0000 7f99f675c640 1 -- 172.21.15.96:0/3896202349 >> 172.21.15.96:0/3896202349 conn(0x7f99f009f2a0 msgr2=0x7f99f009c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:22.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.806+0000 7f99f675c640 1 -- 172.21.15.96:0/3896202349 shutdown_connections 2024-03-21T06:20:22.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.806+0000 7f99f675c640 1 -- 172.21.15.96:0/3896202349 wait complete. 2024-03-21T06:20:22.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.806+0000 7f99f675c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:22.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.807+0000 7f99f675c640 1 Processor -- start 2024-03-21T06:20:22.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.807+0000 7f99f675c640 1 -- start start 2024-03-21T06:20:22.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.808+0000 7f99f675c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99f0099f20 0x7f99f01c3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:22.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.808+0000 7f99f675c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f01299b0 con 0x7f99f0099f20 2024-03-21T06:20:22.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.808+0000 7f99effff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99f0099f20 0x7f99f01c3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:22.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.808+0000 7f99effff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99f0099f20 0x7f99f01c3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35056/0 (socket says 172.21.15.96:35056) 2024-03-21T06:20:22.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.808+0000 7f99effff640 1 -- 172.21.15.96:0/1973774699 learned_addr learned my addr 172.21.15.96:0/1973774699 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:22.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.808+0000 7f99effff640 1 -- 172.21.15.96:0/1973774699 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f01283d0 con 0x7f99f0099f20 2024-03-21T06:20:22.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.809+0000 7f99effff640 1 --2- 172.21.15.96:0/1973774699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99f0099f20 0x7f99f01c3a60 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:22.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.809+0000 7f99ecff9640 1 -- 172.21.15.96:0/1973774699 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f99e0014540 con 0x7f99f0099f20 2024-03-21T06:20:22.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.809+0000 7f99ecff9640 1 -- 172.21.15.96:0/1973774699 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f99e00146a0 con 0x7f99f0099f20 2024-03-21T06:20:22.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.809+0000 7f99f675c640 1 -- 172.21.15.96:0/1973774699 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f01c3770 con 0x7f99f0099f20 2024-03-21T06:20:22.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.809+0000 7f99ecff9640 1 -- 172.21.15.96:0/1973774699 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f99e0014940 con 0x7f99f0099f20 2024-03-21T06:20:22.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.809+0000 7f99f675c640 1 -- 172.21.15.96:0/1973774699 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f0129bc0 con 0x7f99f0099f20 2024-03-21T06:20:22.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.810+0000 7f99ecff9640 1 -- 172.21.15.96:0/1973774699 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) v1 ==== 45326+0+0 (crc 0 0 0) 0x7f99e0014af0 con 0x7f99f0099f20 2024-03-21T06:20:22.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.810+0000 7f99f675c640 1 -- 172.21.15.96:0/1973774699 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f006dc90 con 0x7f99f0099f20 2024-03-21T06:20:22.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.810+0000 7f99ecff9640 1 --2- 172.21.15.96:0/1973774699 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f99dc0381d0 0x7f99dc03a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:22.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.810+0000 7f99ecff9640 1 -- 172.21.15.96:0/1973774699 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2661+0+0 (crc 0 0 0) 0x7f99e0019040 con 0x7f99f0099f20 2024-03-21T06:20:22.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.810+0000 7f99ef7fe640 1 --2- 172.21.15.96:0/1973774699 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f99dc0381d0 0x7f99dc03a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:22.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.811+0000 7f99ef7fe640 1 --2- 172.21.15.96:0/1973774699 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f99dc0381d0 0x7f99dc03a6e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:22.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.815+0000 7f99ecff9640 1 -- 172.21.15.96:0/1973774699 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f99e0018190 con 0x7f99f0099f20 2024-03-21T06:20:22.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:22.993+0000 7f99f675c640 1 -- 172.21.15.96:0/1973774699 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f99f0062f10 con 0x7f99f0099f20 2024-03-21T06:20:23.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.378+0000 7f99ecff9640 1 -- 172.21.15.96:0/1973774699 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f99e0013ca0 con 0x7f99f0099f20 2024-03-21T06:20:23.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.685+0000 7f99ecff9640 1 -- 172.21.15.96:0/1973774699 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v13) v1 ==== 173+0+0 (crc 0 0 0) 0x7f99e0010a00 con 0x7f99f0099f20 2024-03-21T06:20:23.685 INFO:teuthology.orchestra.run.smithi096.stderr:enabled application 'rbd' on pool 'rbd' 2024-03-21T06:20:23.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.687+0000 7f99f675c640 1 -- 172.21.15.96:0/1973774699 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f99dc0381d0 msgr2=0x7f99dc03a6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:23.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.687+0000 7f99f675c640 1 --2- 172.21.15.96:0/1973774699 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f99dc0381d0 0x7f99dc03a6e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:23.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.687+0000 7f99f675c640 1 -- 172.21.15.96:0/1973774699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99f0099f20 msgr2=0x7f99f01c3a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:23.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.687+0000 7f99f675c640 1 --2- 172.21.15.96:0/1973774699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99f0099f20 0x7f99f01c3a60 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:23.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.687+0000 7f99f675c640 1 -- 172.21.15.96:0/1973774699 shutdown_connections 2024-03-21T06:20:23.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.688+0000 7f99f675c640 1 --2- 172.21.15.96:0/1973774699 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f99dc0381d0 0x7f99dc03a6e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:23.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.688+0000 7f99f675c640 1 --2- 172.21.15.96:0/1973774699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99f0099f20 0x7f99f01c3a60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:23.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.688+0000 7f99f675c640 1 -- 172.21.15.96:0/1973774699 >> 172.21.15.96:0/1973774699 conn(0x7f99f009f2a0 msgr2=0x7f99f012f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:23.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.688+0000 7f99f675c640 1 -- 172.21.15.96:0/1973774699 shutdown_connections 2024-03-21T06:20:23.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.688+0000 7f99f675c640 1 -- 172.21.15.96:0/1973774699 wait complete. 2024-03-21T06:20:23.705 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-21T06:20:23.706 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-21T06:20:23.706 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-21T06:20:23.706 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-21T06:20:23.707 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-21T06:20:23.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.816+0000 7fabc04dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:23.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.820+0000 7fabc04dc640 1 Processor -- start 2024-03-21T06:20:23.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.820+0000 7fabc04dc640 1 -- start start 2024-03-21T06:20:23.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.821+0000 7fabc04dc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fabb809cda0 0x7fabb8139450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:23.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.821+0000 7fabc04dc640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb806e410 con 0x7fabb809b360 2024-03-21T06:20:23.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.821+0000 7fabc04dc640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb809b770 con 0x7fabb809cda0 2024-03-21T06:20:23.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.821+0000 7fabbda77640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fabb809cda0 0x7fabb8139450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:23.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.821+0000 7fabbe278640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fabb809b360 0x7fabb81315d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33070/0 (socket says 172.21.15.96:33070) 2024-03-21T06:20:23.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.821+0000 7fabbe278640 1 -- 172.21.15.96:0/1346354775 learned_addr learned my addr 172.21.15.96:0/1346354775 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:23.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.822+0000 7fabbda77640 1 -- 172.21.15.96:0/1346354775 >> v1:172.21.15.96:6789/0 conn(0x7fabb809b360 legacy=0x7fabb81315d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:23.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.822+0000 7fabbda77640 1 -- 172.21.15.96:0/1346354775 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb809d1b0 con 0x7fabb809cda0 2024-03-21T06:20:23.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.822+0000 7fabbda77640 1 --2- 172.21.15.96:0/1346354775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabb809cda0 0x7fabb8139450 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a796d784a33f40 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:23.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.822+0000 7fabbd276640 1 -- 172.21.15.96:0/1346354775 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7faba8009ce0 con 0x7fabb809cda0 2024-03-21T06:20:23.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.822+0000 7fabbd276640 1 -- 172.21.15.96:0/1346354775 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7faba8009e40 con 0x7fabb809cda0 2024-03-21T06:20:23.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.822+0000 7fabbd276640 1 -- 172.21.15.96:0/1346354775 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7faba800e360 con 0x7fabb809cda0 2024-03-21T06:20:23.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.822+0000 7fabc04dc640 1 -- 172.21.15.96:0/1346354775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabb809cda0 msgr2=0x7fabb8139450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:23.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.822+0000 7fabc04dc640 1 --2- 172.21.15.96:0/1346354775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabb809cda0 0x7fabb8139450 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:23.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.822+0000 7fabc04dc640 1 -- 172.21.15.96:0/1346354775 shutdown_connections 2024-03-21T06:20:23.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.822+0000 7fabc04dc640 1 --2- 172.21.15.96:0/1346354775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabb809cda0 0x7fabb8139450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:23.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.823+0000 7fabc04dc640 1 -- 172.21.15.96:0/1346354775 >> 172.21.15.96:0/1346354775 conn(0x7fabb80a42c0 msgr2=0x7fabb80a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:23.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.823+0000 7fabc04dc640 1 -- 172.21.15.96:0/1346354775 shutdown_connections 2024-03-21T06:20:23.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.823+0000 7fabc04dc640 1 -- 172.21.15.96:0/1346354775 wait complete. 2024-03-21T06:20:23.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.823+0000 7fabc04dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:23.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.824+0000 7fabc04dc640 1 Processor -- start 2024-03-21T06:20:23.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.824+0000 7fabc04dc640 1 -- start start 2024-03-21T06:20:23.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.824+0000 7fabc04dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabb809b360 0x7fabb81c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:23.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.824+0000 7fabc04dc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb811e420 con 0x7fabb809b360 2024-03-21T06:20:23.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.824+0000 7fabbe278640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabb809b360 0x7fabb81c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:23.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.824+0000 7fabbe278640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabb809b360 0x7fabb81c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35068/0 (socket says 172.21.15.96:35068) 2024-03-21T06:20:23.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.824+0000 7fabbe278640 1 -- 172.21.15.96:0/3714775808 learned_addr learned my addr 172.21.15.96:0/3714775808 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:23.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.825+0000 7fabbe278640 1 -- 172.21.15.96:0/3714775808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb8073030 con 0x7fabb809b360 2024-03-21T06:20:23.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.825+0000 7fabbe278640 1 --2- 172.21.15.96:0/3714775808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabb809b360 0x7fabb81c8d20 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:23.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.825+0000 7fabaeffd640 1 -- 172.21.15.96:0/3714775808 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fabb40070b0 con 0x7fabb809b360 2024-03-21T06:20:23.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.825+0000 7fabaeffd640 1 -- 172.21.15.96:0/3714775808 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fabb4007210 con 0x7fabb809b360 2024-03-21T06:20:23.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.825+0000 7fabc04dc640 1 -- 172.21.15.96:0/3714775808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb811cd30 con 0x7fabb809b360 2024-03-21T06:20:23.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.825+0000 7fabaeffd640 1 -- 172.21.15.96:0/3714775808 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fabb4007380 con 0x7fabb809b360 2024-03-21T06:20:23.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.825+0000 7fabc04dc640 1 -- 172.21.15.96:0/3714775808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb811d510 con 0x7fabb809b360 2024-03-21T06:20:23.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.826+0000 7fabaeffd640 1 -- 172.21.15.96:0/3714775808 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fabb4007530 con 0x7fabb809b360 2024-03-21T06:20:23.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.826+0000 7fabc04dc640 1 -- 172.21.15.96:0/3714775808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb811cf20 con 0x7fabb809b360 2024-03-21T06:20:23.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.826+0000 7fabaeffd640 1 --2- 172.21.15.96:0/3714775808 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fab9c038260 0x7fab9c03a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:23.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.826+0000 7fabaeffd640 1 -- 172.21.15.96:0/3714775808 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fabb40480c0 con 0x7fabb809b360 2024-03-21T06:20:23.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.826+0000 7fabbda77640 1 --2- 172.21.15.96:0/3714775808 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fab9c038260 0x7fab9c03a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:23.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.827+0000 7fabbda77640 1 --2- 172.21.15.96:0/3714775808 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fab9c038260 0x7fab9c03a770 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:23.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:23.831+0000 7fabaeffd640 1 -- 172.21.15.96:0/3714775808 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fabb811cf20 con 0x7fabb809b360 2024-03-21T06:20:24.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.052+0000 7fabc04dc640 1 -- 172.21.15.96:0/3714775808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fabb8073030 con 0x7fabb809b360 2024-03-21T06:20:24.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.054+0000 7fabaeffd640 1 -- 172.21.15.96:0/3714775808 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+87025 (crc 0 0 0) 0x7fabb8073030 con 0x7fabb809b360 2024-03-21T06:20:24.056 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:20:24.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.056+0000 7fabc04dc640 1 -- 172.21.15.96:0/3714775808 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fab9c038260 msgr2=0x7fab9c03a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.056+0000 7fabc04dc640 1 --2- 172.21.15.96:0/3714775808 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fab9c038260 0x7fab9c03a770 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.056+0000 7fabc04dc640 1 -- 172.21.15.96:0/3714775808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabb809b360 msgr2=0x7fabb81c8d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.056+0000 7fabc04dc640 1 --2- 172.21.15.96:0/3714775808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabb809b360 0x7fabb81c8d20 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.056+0000 7fabc04dc640 1 -- 172.21.15.96:0/3714775808 shutdown_connections 2024-03-21T06:20:24.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.057+0000 7fabc04dc640 1 --2- 172.21.15.96:0/3714775808 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fab9c038260 0x7fab9c03a770 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.057+0000 7fabc04dc640 1 --2- 172.21.15.96:0/3714775808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fabb809b360 0x7fabb81c8d20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.057+0000 7fabc04dc640 1 -- 172.21.15.96:0/3714775808 >> 172.21.15.96:0/3714775808 conn(0x7fabb80a42c0 msgr2=0x7fabb80a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:24.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.057+0000 7fabc04dc640 1 -- 172.21.15.96:0/3714775808 shutdown_connections 2024-03-21T06:20:24.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.057+0000 7fabc04dc640 1 -- 172.21.15.96:0/3714775808 wait complete. 2024-03-21T06:20:24.067 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2376803682},{"type":"v1","addr":"172.21.15.184:6801","nonce":2376803682}]},"active_addr":"172.21.15.184:6801/2376803682","active_change":"2024-03-21T06:20:16.606137+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":159888069}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":3959660977}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":537859240}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.184:0","nonce":3390473790}]}]} 2024-03-21T06:20:24.069 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-21T06:20:24.069 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-21T06:20:24.070 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T06:20:24.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.172+0000 7f565ba24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:24.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.176+0000 7f565ba24640 1 Processor -- start 2024-03-21T06:20:24.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.176+0000 7f565ba24640 1 -- start start 2024-03-21T06:20:24.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.177+0000 7f565ba24640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f565406d9e0 0x7f565409ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.177+0000 7f565ba24640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5654099960 con 0x7f565409b290 2024-03-21T06:20:24.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.177+0000 7f565ba24640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f565409bf80 con 0x7f565406d9e0 2024-03-21T06:20:24.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.177+0000 7f56597c0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f565406d9e0 0x7f565409ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.177+0000 7f5658fbf640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f565409b290 0x7f5654098090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33080/0 (socket says 172.21.15.96:33080) 2024-03-21T06:20:24.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.177+0000 7f5658fbf640 1 -- 172.21.15.96:0/1646203938 learned_addr learned my addr 172.21.15.96:0/1646203938 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:24.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.178+0000 7f56597c0640 1 -- 172.21.15.96:0/1646203938 >> v1:172.21.15.96:6789/0 conn(0x7f565409b290 legacy=0x7f5654098090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.178+0000 7f56597c0640 1 -- 172.21.15.96:0/1646203938 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565409c1c0 con 0x7f565406d9e0 2024-03-21T06:20:24.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.178+0000 7f56597c0640 1 --2- 172.21.15.96:0/1646203938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565406d9e0 0x7f565409ad50 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537f696e55c961b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.178+0000 7f564bfff640 1 -- 172.21.15.96:0/1646203938 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f565000e040 con 0x7f565406d9e0 2024-03-21T06:20:24.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.178+0000 7f564bfff640 1 -- 172.21.15.96:0/1646203938 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5650007670 con 0x7f565406d9e0 2024-03-21T06:20:24.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.178+0000 7f564bfff640 1 -- 172.21.15.96:0/1646203938 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5650007910 con 0x7f565406d9e0 2024-03-21T06:20:24.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.178+0000 7f565ba24640 1 -- 172.21.15.96:0/1646203938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565406d9e0 msgr2=0x7f565409ad50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.178+0000 7f565ba24640 1 --2- 172.21.15.96:0/1646203938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565406d9e0 0x7f565409ad50 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.178+0000 7f565ba24640 1 -- 172.21.15.96:0/1646203938 shutdown_connections 2024-03-21T06:20:24.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.178+0000 7f565ba24640 1 --2- 172.21.15.96:0/1646203938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565406d9e0 0x7f565409ad50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.178+0000 7f565ba24640 1 -- 172.21.15.96:0/1646203938 >> 172.21.15.96:0/1646203938 conn(0x7f565409fdb0 msgr2=0x7f56540a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:24.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.179+0000 7f565ba24640 1 -- 172.21.15.96:0/1646203938 shutdown_connections 2024-03-21T06:20:24.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.179+0000 7f565ba24640 1 -- 172.21.15.96:0/1646203938 wait complete. 2024-03-21T06:20:24.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.179+0000 7f565ba24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:24.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.180+0000 7f565ba24640 1 Processor -- start 2024-03-21T06:20:24.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.180+0000 7f565ba24640 1 -- start start 2024-03-21T06:20:24.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.180+0000 7f565ba24640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565409b290 0x7f565408bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.180+0000 7f565ba24640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56541319e0 con 0x7f565409b290 2024-03-21T06:20:24.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.180+0000 7f56597c0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565409b290 0x7f565408bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.180+0000 7f56597c0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565409b290 0x7f565408bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35096/0 (socket says 172.21.15.96:35096) 2024-03-21T06:20:24.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.180+0000 7f56597c0640 1 -- 172.21.15.96:0/891569251 learned_addr learned my addr 172.21.15.96:0/891569251 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:24.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.181+0000 7f56597c0640 1 -- 172.21.15.96:0/891569251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565408df70 con 0x7f565409b290 2024-03-21T06:20:24.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.181+0000 7f56597c0640 1 --2- 172.21.15.96:0/891569251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565409b290 0x7f565408bfd0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.181+0000 7f5649ffb640 1 -- 172.21.15.96:0/891569251 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f565000e040 con 0x7f565409b290 2024-03-21T06:20:24.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.181+0000 7f5649ffb640 1 -- 172.21.15.96:0/891569251 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5650012a90 con 0x7f565409b290 2024-03-21T06:20:24.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.181+0000 7f565ba24640 1 -- 172.21.15.96:0/891569251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5654073990 con 0x7f565409b290 2024-03-21T06:20:24.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.181+0000 7f5649ffb640 1 -- 172.21.15.96:0/891569251 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5650012d30 con 0x7f565409b290 2024-03-21T06:20:24.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.181+0000 7f565ba24640 1 -- 172.21.15.96:0/891569251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f565408d9b0 con 0x7f565409b290 2024-03-21T06:20:24.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.182+0000 7f5649ffb640 1 -- 172.21.15.96:0/891569251 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f56500153f0 con 0x7f565409b290 2024-03-21T06:20:24.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.182+0000 7f5649ffb640 1 --2- 172.21.15.96:0/891569251 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5638038260 0x7f563803a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.182+0000 7f5649ffb640 1 -- 172.21.15.96:0/891569251 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f56500490a0 con 0x7f565409b290 2024-03-21T06:20:24.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.182+0000 7f565ba24640 1 -- 172.21.15.96:0/891569251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5654073990 con 0x7f565409b290 2024-03-21T06:20:24.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.182+0000 7f5658fbf640 1 --2- 172.21.15.96:0/891569251 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5638038260 0x7f563803a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.183+0000 7f5658fbf640 1 --2- 172.21.15.96:0/891569251 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5638038260 0x7f563803a770 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.187+0000 7f5649ffb640 1 -- 172.21.15.96:0/891569251 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f5654073990 con 0x7f565409b290 2024-03-21T06:20:24.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.358+0000 7f565ba24640 1 -- 172.21.15.96:0/891569251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5654073990 con 0x7f565409b290 2024-03-21T06:20:24.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.359+0000 7f5649ffb640 1 -- 172.21.15.96:0/891569251 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+8699 (crc 0 0 0) 0x7f5654073990 con 0x7f565409b290 2024-03-21T06:20:24.359 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:20:24.359 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":13,"fsid":"e880df84-2d7c-4386-b5e9-f3879b9ab9d5","created":"2024-03-21T06:20:12.270291+0000","modified":"2024-03-21T06:20:23.677980+0000","last_up_change":"2024-03-21T06:20:18.657108+0000","last_in_change":"2024-03-21T06:20:14.428643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T06:20:20.651568+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-21T06:20:22.673961+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffab25be-1f6f-450d-a8b4-da7f86f3eaeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6801","nonce":932229279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":932229279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":932229279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6803","nonce":932229279}]},"public_addr":"172.21.15.96:6801/932229279","cluster_addr":"172.21.15.96:6802/932229279","heartbeat_back_addr":"172.21.15.96:6808/932229279","heartbeat_front_addr":"172.21.15.96:6803/932229279","state":["exists","up"]},{"osd":1,"uuid":"3266cd80-6286-4a69-933a-00cd48d67487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6805","nonce":13988474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":13988474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":13988474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6807","nonce":13988474}]},"public_addr":"172.21.15.96:6805/13988474","cluster_addr":"172.21.15.96:6804/13988474","heartbeat_back_addr":"172.21.15.96:6806/13988474","heartbeat_front_addr":"172.21.15.96:6807/13988474","state":["exists","up"]},{"osd":2,"uuid":"5a8b1e5a-d4ed-4c96-b301-4173028235d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":155456265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":155456265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":155456265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6815","nonce":155456265}]},"public_addr":"172.21.15.184:6804/155456265","cluster_addr":"172.21.15.184:6805/155456265","heartbeat_back_addr":"172.21.15.184:6814/155456265","heartbeat_front_addr":"172.21.15.184:6815/155456265","state":["exists","up"]},{"osd":3,"uuid":"5945ad82-8437-4490-aca6-9c454678cd34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":276480837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":276480837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":276480837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":276480837}]},"public_addr":"172.21.15.184:6811/276480837","cluster_addr":"172.21.15.184:6810/276480837","heartbeat_back_addr":"172.21.15.184:6808/276480837","heartbeat_front_addr":"172.21.15.184:6809/276480837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T06:20:24.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.361+0000 7f565ba24640 1 -- 172.21.15.96:0/891569251 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5638038260 msgr2=0x7f563803a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.361+0000 7f565ba24640 1 --2- 172.21.15.96:0/891569251 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5638038260 0x7f563803a770 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.361+0000 7f565ba24640 1 -- 172.21.15.96:0/891569251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565409b290 msgr2=0x7f565408bfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.361+0000 7f565ba24640 1 --2- 172.21.15.96:0/891569251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565409b290 0x7f565408bfd0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.361+0000 7f565ba24640 1 -- 172.21.15.96:0/891569251 shutdown_connections 2024-03-21T06:20:24.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.361+0000 7f565ba24640 1 --2- 172.21.15.96:0/891569251 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5638038260 0x7f563803a770 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.361+0000 7f565ba24640 1 --2- 172.21.15.96:0/891569251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f565409b290 0x7f565408bfd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.361+0000 7f565ba24640 1 -- 172.21.15.96:0/891569251 >> 172.21.15.96:0/891569251 conn(0x7f565409fdb0 msgr2=0x7f56540a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:24.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.361+0000 7f565ba24640 1 -- 172.21.15.96:0/891569251 shutdown_connections 2024-03-21T06:20:24.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.361+0000 7f565ba24640 1 -- 172.21.15.96:0/891569251 wait complete. 2024-03-21T06:20:24.372 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-21T06:20:24.372 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T06:20:24.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.473+0000 7f122cc24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:24.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.477+0000 7f122cc24640 1 Processor -- start 2024-03-21T06:20:24.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.477+0000 7f122cc24640 1 -- start start 2024-03-21T06:20:24.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.478+0000 7f122cc24640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f122809b460 0x7f1228136460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.478+0000 7f122cc24640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1228099960 con 0x7f122806d9e0 2024-03-21T06:20:24.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.478+0000 7f122cc24640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122809bf80 con 0x7f122809b460 2024-03-21T06:20:24.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.478+0000 7f122659c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f122806d9e0 0x7f122809ad50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33094/0 (socket says 172.21.15.96:33094) 2024-03-21T06:20:24.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.478+0000 7f122659c640 1 -- 172.21.15.96:0/3023671653 learned_addr learned my addr 172.21.15.96:0/3023671653 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:24.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.478+0000 7f1225d9b640 1 --2- 172.21.15.96:0/3023671653 >> v2:172.21.15.96:3300/0 conn(0x7f122809b460 0x7f1228136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.478+0000 7f122559a640 1 -- 172.21.15.96:0/3023671653 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404521296 0 0) 0x7f1228099960 con 0x7f122806d9e0 2024-03-21T06:20:24.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.478+0000 7f122559a640 1 -- 172.21.15.96:0/3023671653 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12100034f0 con 0x7f122806d9e0 2024-03-21T06:20:24.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.478+0000 7f1225d9b640 1 -- 172.21.15.96:0/3023671653 >> v1:172.21.15.96:6789/0 conn(0x7f122806d9e0 legacy=0x7f122809ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.478+0000 7f1225d9b640 1 -- 172.21.15.96:0/3023671653 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122809c1c0 con 0x7f122809b460 2024-03-21T06:20:24.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.478+0000 7f1225d9b640 1 --2- 172.21.15.96:0/3023671653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f122809b460 0x7f1228136460 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bca6b41fd682203 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.479+0000 7f122559a640 1 -- 172.21.15.96:0/3023671653 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f121c009ce0 con 0x7f122809b460 2024-03-21T06:20:24.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.479+0000 7f122559a640 1 -- 172.21.15.96:0/3023671653 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f121c009e40 con 0x7f122809b460 2024-03-21T06:20:24.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.479+0000 7f122559a640 1 -- 172.21.15.96:0/3023671653 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f121c00e360 con 0x7f122809b460 2024-03-21T06:20:24.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.479+0000 7f122cc24640 1 -- 172.21.15.96:0/3023671653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f122809b460 msgr2=0x7f1228136460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.479+0000 7f122cc24640 1 --2- 172.21.15.96:0/3023671653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f122809b460 0x7f1228136460 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.479+0000 7f122cc24640 1 -- 172.21.15.96:0/3023671653 shutdown_connections 2024-03-21T06:20:24.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.479+0000 7f122cc24640 1 --2- 172.21.15.96:0/3023671653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f122809b460 0x7f1228136460 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.479+0000 7f122cc24640 1 -- 172.21.15.96:0/3023671653 >> 172.21.15.96:0/3023671653 conn(0x7f122809fdb0 msgr2=0x7f12280a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:24.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.479+0000 7f122cc24640 1 -- 172.21.15.96:0/3023671653 shutdown_connections 2024-03-21T06:20:24.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.479+0000 7f122cc24640 1 -- 172.21.15.96:0/3023671653 wait complete. 2024-03-21T06:20:24.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.480+0000 7f122cc24640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:24.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.481+0000 7f122cc24640 1 Processor -- start 2024-03-21T06:20:24.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.481+0000 7f122cc24640 1 -- start start 2024-03-21T06:20:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.481+0000 7f122cc24640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f122809b460 0x7f1228130ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.481+0000 7f122cc24640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1228098b50 con 0x7f122809b460 2024-03-21T06:20:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.481+0000 7f122659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f122809b460 0x7f1228130ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.481+0000 7f122659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f122809b460 0x7f1228130ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35122/0 (socket says 172.21.15.96:35122) 2024-03-21T06:20:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.481+0000 7f122659c640 1 -- 172.21.15.96:0/411754591 learned_addr learned my addr 172.21.15.96:0/411754591 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.482+0000 7f122659c640 1 -- 172.21.15.96:0/411754591 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1228130190 con 0x7f122809b460 2024-03-21T06:20:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.482+0000 7f122659c640 1 --2- 172.21.15.96:0/411754591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f122809b460 0x7f1228130ac0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.482+0000 7f1216ffd640 1 -- 172.21.15.96:0/411754591 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f12180059d0 con 0x7f122809b460 2024-03-21T06:20:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.482+0000 7f1216ffd640 1 -- 172.21.15.96:0/411754591 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f1218005b30 con 0x7f122809b460 2024-03-21T06:20:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.482+0000 7f122cc24640 1 -- 172.21.15.96:0/411754591 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12281303c0 con 0x7f122809b460 2024-03-21T06:20:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.482+0000 7f1216ffd640 1 -- 172.21.15.96:0/411754591 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f1218005dd0 con 0x7f122809b460 2024-03-21T06:20:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.482+0000 7f122cc24640 1 -- 172.21.15.96:0/411754591 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122812f030 con 0x7f122809b460 2024-03-21T06:20:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.483+0000 7f1216ffd640 1 -- 172.21.15.96:0/411754591 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f12180070b0 con 0x7f122809b460 2024-03-21T06:20:24.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.483+0000 7f122cc24640 1 -- 172.21.15.96:0/411754591 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12281305b0 con 0x7f122809b460 2024-03-21T06:20:24.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.483+0000 7f1216ffd640 1 --2- 172.21.15.96:0/411754591 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f1208038260 0x7f120803a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.483+0000 7f1216ffd640 1 -- 172.21.15.96:0/411754591 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f1218047850 con 0x7f122809b460 2024-03-21T06:20:24.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.483+0000 7f1225d9b640 1 --2- 172.21.15.96:0/411754591 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f1208038260 0x7f120803a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.484+0000 7f1225d9b640 1 --2- 172.21.15.96:0/411754591 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f1208038260 0x7f120803a770 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.488+0000 7f1216ffd640 1 -- 172.21.15.96:0/411754591 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f121800d510 con 0x7f122809b460 2024-03-21T06:20:24.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.662+0000 7f122cc24640 1 -- 172.21.15.96:0/411754591 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1228062f10 con 0x7f122809b460 2024-03-21T06:20:24.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.663+0000 7f1216ffd640 1 -- 172.21.15.96:0/411754591 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+8699 (crc 0 0 0) 0x7f121800d150 con 0x7f122809b460 2024-03-21T06:20:24.663 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:20:24.663 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":13,"fsid":"e880df84-2d7c-4386-b5e9-f3879b9ab9d5","created":"2024-03-21T06:20:12.270291+0000","modified":"2024-03-21T06:20:23.677980+0000","last_up_change":"2024-03-21T06:20:18.657108+0000","last_in_change":"2024-03-21T06:20:14.428643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T06:20:20.651568+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-21T06:20:22.673961+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffab25be-1f6f-450d-a8b4-da7f86f3eaeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6801","nonce":932229279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":932229279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":932229279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6803","nonce":932229279}]},"public_addr":"172.21.15.96:6801/932229279","cluster_addr":"172.21.15.96:6802/932229279","heartbeat_back_addr":"172.21.15.96:6808/932229279","heartbeat_front_addr":"172.21.15.96:6803/932229279","state":["exists","up"]},{"osd":1,"uuid":"3266cd80-6286-4a69-933a-00cd48d67487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6805","nonce":13988474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":13988474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":13988474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6807","nonce":13988474}]},"public_addr":"172.21.15.96:6805/13988474","cluster_addr":"172.21.15.96:6804/13988474","heartbeat_back_addr":"172.21.15.96:6806/13988474","heartbeat_front_addr":"172.21.15.96:6807/13988474","state":["exists","up"]},{"osd":2,"uuid":"5a8b1e5a-d4ed-4c96-b301-4173028235d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":155456265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":155456265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":155456265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6815","nonce":155456265}]},"public_addr":"172.21.15.184:6804/155456265","cluster_addr":"172.21.15.184:6805/155456265","heartbeat_back_addr":"172.21.15.184:6814/155456265","heartbeat_front_addr":"172.21.15.184:6815/155456265","state":["exists","up"]},{"osd":3,"uuid":"5945ad82-8437-4490-aca6-9c454678cd34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":276480837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":276480837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":276480837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":276480837}]},"public_addr":"172.21.15.184:6811/276480837","cluster_addr":"172.21.15.184:6810/276480837","heartbeat_back_addr":"172.21.15.184:6808/276480837","heartbeat_front_addr":"172.21.15.184:6809/276480837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T06:20:24.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.665+0000 7f122cc24640 1 -- 172.21.15.96:0/411754591 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f1208038260 msgr2=0x7f120803a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.665+0000 7f122cc24640 1 --2- 172.21.15.96:0/411754591 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f1208038260 0x7f120803a770 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.665+0000 7f122cc24640 1 -- 172.21.15.96:0/411754591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f122809b460 msgr2=0x7f1228130ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.665+0000 7f122cc24640 1 --2- 172.21.15.96:0/411754591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f122809b460 0x7f1228130ac0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.665+0000 7f122cc24640 1 -- 172.21.15.96:0/411754591 shutdown_connections 2024-03-21T06:20:24.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.665+0000 7f122cc24640 1 --2- 172.21.15.96:0/411754591 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f1208038260 0x7f120803a770 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.665+0000 7f122cc24640 1 --2- 172.21.15.96:0/411754591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f122809b460 0x7f1228130ac0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.665+0000 7f122cc24640 1 -- 172.21.15.96:0/411754591 >> 172.21.15.96:0/411754591 conn(0x7f122809fdb0 msgr2=0x7f1228126f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:24.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.665+0000 7f122cc24640 1 -- 172.21.15.96:0/411754591 shutdown_connections 2024-03-21T06:20:24.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.665+0000 7f122cc24640 1 -- 172.21.15.96:0/411754591 wait complete. 2024-03-21T06:20:24.677 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-21T06:20:24.677 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-21T06:20:24.678 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-21T06:20:24.678 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-21T06:20:24.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.790+0000 7fa0d3cdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:24.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.790+0000 7fc25cf9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:24.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.790+0000 7f739e264640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:24.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.790+0000 7fb3dd15c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:24.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.795+0000 7fc25cf9c640 1 Processor -- start 2024-03-21T06:20:24.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.795+0000 7fa0d3cdc640 1 Processor -- start 2024-03-21T06:20:24.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.795+0000 7fc25cf9c640 1 -- start start 2024-03-21T06:20:24.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.795+0000 7fb3dd15c640 1 Processor -- start 2024-03-21T06:20:24.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.795+0000 7fb3dd15c640 1 -- start start 2024-03-21T06:20:24.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.795+0000 7fa0d3cdc640 1 -- start start 2024-03-21T06:20:24.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.795+0000 7f739e264640 1 Processor -- start 2024-03-21T06:20:24.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.795+0000 7f739e264640 1 -- start start 2024-03-21T06:20:24.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.795+0000 7fc25cf9c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc25809b5d0 0x7fc258130460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fc25cf9c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc258099dc0 con 0x7fc2581309a0 2024-03-21T06:20:24.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fc25cf9c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc25809bae0 con 0x7fc25809b5d0 2024-03-21T06:20:24.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fc25659c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc25809b5d0 0x7fc258130460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fb3dd15c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb3d8137540 0x7fb3d8139960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fb3dd15c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d806e260 con 0x7fb3d809b650 2024-03-21T06:20:24.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fb3dd15c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d809abb0 con 0x7fb3d8137540 2024-03-21T06:20:24.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fc25659c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc25809b5d0 0x7fc258130460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35138/0 (socket says 172.21.15.96:35138) 2024-03-21T06:20:24.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fc25659c640 1 -- 172.21.15.96:0/4274499220 learned_addr learned my addr 172.21.15.96:0/4274499220 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:24.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fb3d659c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb3d8137540 0x7fb3d8139960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fb3d6d9d640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fb3d809b650 0x7fb3d806d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33120/0 (socket says 172.21.15.96:33120) 2024-03-21T06:20:24.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fb3d6d9d640 1 -- 172.21.15.96:0/4240415753 learned_addr learned my addr 172.21.15.96:0/4240415753 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:24.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fc25659c640 1 -- 172.21.15.96:0/4274499220 >> v1:172.21.15.96:6789/0 conn(0x7fc2581309a0 legacy=0x7fc258098190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fa0d3cdc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa0cc09b360 0x7fa0cc09cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fc255d9b640 1 -- 172.21.15.96:0/4274499220 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388372292 0 0) 0x7fc258099dc0 con 0x7fc2581309a0 2024-03-21T06:20:24.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fc25659c640 1 -- 172.21.15.96:0/4274499220 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25809c0f0 con 0x7fc25809b5d0 2024-03-21T06:20:24.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fb3d659c640 1 -- 172.21.15.96:0/4240415753 >> v1:172.21.15.96:6789/0 conn(0x7fb3d809b650 legacy=0x7fb3d806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fb3d659c640 1 -- 172.21.15.96:0/4240415753 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d809ba60 con 0x7fb3d8137540 2024-03-21T06:20:24.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fa0d3cdc640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0cc06e410 con 0x7fa0cc1315d0 2024-03-21T06:20:24.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fa0d3cdc640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0cc09b770 con 0x7fa0cc09b360 2024-03-21T06:20:24.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fa0d1a78640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa0cc09b360 0x7fa0cc09cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fa0d1277640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fa0cc1315d0 0x7fa0cc134130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33136/0 (socket says 172.21.15.96:33136) 2024-03-21T06:20:24.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fa0d1277640 1 -- 172.21.15.96:0/21656460 learned_addr learned my addr 172.21.15.96:0/21656460 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:24.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.796+0000 7fb3d659c640 1 --2- 172.21.15.96:0/4240415753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d8137540 0x7fb3d8139960 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb899c7880dca812 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7f739e264640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f739806e820 0x7f7398071c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7f739e264640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739806f1e0 con 0x7f73980721d0 2024-03-21T06:20:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7f739e264640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f739806f480 con 0x7f739806e820 2024-03-21T06:20:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7f7397fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f739806e820 0x7f7398071c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7f7397fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f739806e820 0x7f7398071c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35150/0 (socket says 172.21.15.96:35150) 2024-03-21T06:20:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7f7397fff640 1 -- 172.21.15.96:0/3489466914 learned_addr learned my addr 172.21.15.96:0/3489466914 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:24.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7fc25659c640 1 --2- 172.21.15.96:0/4274499220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc25809b5d0 0x7fc258130460 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5c0c4616e5063e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7fb3d5d9b640 1 -- 172.21.15.96:0/4240415753 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb3cc009ce0 con 0x7fb3d8137540 2024-03-21T06:20:24.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7fa0d0a76640 1 -- 172.21.15.96:0/21656460 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882180687 0 0) 0x7fa0cc06e410 con 0x7fa0cc1315d0 2024-03-21T06:20:24.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7fa0d1a78640 1 -- 172.21.15.96:0/21656460 >> v1:172.21.15.96:6789/0 conn(0x7fa0cc1315d0 legacy=0x7fa0cc134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7fa0d1a78640 1 -- 172.21.15.96:0/21656460 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc06dd00 con 0x7fa0cc09b360 2024-03-21T06:20:24.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7fb3d5d9b640 1 -- 172.21.15.96:0/4240415753 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb3cc009e40 con 0x7fb3d8137540 2024-03-21T06:20:24.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7fb3d5d9b640 1 -- 172.21.15.96:0/4240415753 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb3cc00e360 con 0x7fb3d8137540 2024-03-21T06:20:24.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7fa0d1a78640 1 --2- 172.21.15.96:0/21656460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0cc09b360 0x7fa0cc09cda0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c01b29816e21860 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7f7396ffd640 1 -- 172.21.15.96:0/3489466914 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3780470005 0 0) 0x7f739806f1e0 con 0x7f73980721d0 2024-03-21T06:20:24.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7f7396ffd640 1 -- 172.21.15.96:0/3489466914 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73800034f0 con 0x7f73980721d0 2024-03-21T06:20:24.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7f7397fff640 1 -- 172.21.15.96:0/3489466914 >> v1:172.21.15.96:6789/0 conn(0x7f73980721d0 legacy=0x7f739809e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7fc255d9b640 1 -- 172.21.15.96:0/4274499220 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc248009dd0 con 0x7fc25809b5d0 2024-03-21T06:20:24.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7f7397fff640 1 -- 172.21.15.96:0/3489466914 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739809f860 con 0x7f739806e820 2024-03-21T06:20:24.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7fa0d0a76640 1 -- 172.21.15.96:0/21656460 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa0c8009ce0 con 0x7fa0cc09b360 2024-03-21T06:20:24.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7fa0d0a76640 1 -- 172.21.15.96:0/21656460 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa0c8009e40 con 0x7fa0cc09b360 2024-03-21T06:20:24.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7fa0d0a76640 1 -- 172.21.15.96:0/21656460 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa0c800e360 con 0x7fa0cc09b360 2024-03-21T06:20:24.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7fc255d9b640 1 -- 172.21.15.96:0/4274499220 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc24800e230 con 0x7fc25809b5d0 2024-03-21T06:20:24.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7fc255d9b640 1 -- 172.21.15.96:0/4274499220 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc24800e4d0 con 0x7fc25809b5d0 2024-03-21T06:20:24.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.797+0000 7f7397fff640 1 --2- 172.21.15.96:0/3489466914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f739806e820 0x7f7398071c90 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351ef5cdb2bc3156 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7f7396ffd640 1 -- 172.21.15.96:0/3489466914 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7388009da0 con 0x7f739806e820 2024-03-21T06:20:24.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7f7396ffd640 1 -- 172.21.15.96:0/3489466914 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f738800e230 con 0x7f739806e820 2024-03-21T06:20:24.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7f7396ffd640 1 -- 172.21.15.96:0/3489466914 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f738800e4d0 con 0x7f739806e820 2024-03-21T06:20:24.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fb3dd15c640 1 -- 172.21.15.96:0/4240415753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d8137540 msgr2=0x7fb3d8139960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fb3dd15c640 1 --2- 172.21.15.96:0/4240415753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d8137540 0x7fb3d8139960 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fb3dd15c640 1 -- 172.21.15.96:0/4240415753 shutdown_connections 2024-03-21T06:20:24.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fb3dd15c640 1 --2- 172.21.15.96:0/4240415753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d8137540 0x7fb3d8139960 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fb3dd15c640 1 -- 172.21.15.96:0/4240415753 >> 172.21.15.96:0/4240415753 conn(0x7fb3d80a45c0 msgr2=0x7fb3d80a49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:24.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fb3dd15c640 1 -- 172.21.15.96:0/4240415753 shutdown_connections 2024-03-21T06:20:24.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fb3dd15c640 1 -- 172.21.15.96:0/4240415753 wait complete. 2024-03-21T06:20:24.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fa0d3cdc640 1 -- 172.21.15.96:0/21656460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0cc09b360 msgr2=0x7fa0cc09cda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fa0d3cdc640 1 --2- 172.21.15.96:0/21656460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0cc09b360 0x7fa0cc09cda0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fa0d3cdc640 1 -- 172.21.15.96:0/21656460 shutdown_connections 2024-03-21T06:20:24.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fa0d3cdc640 1 --2- 172.21.15.96:0/21656460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0cc09b360 0x7fa0cc09cda0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fb3dd15c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:24.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fa0d3cdc640 1 -- 172.21.15.96:0/21656460 >> 172.21.15.96:0/21656460 conn(0x7fa0cc0a42c0 msgr2=0x7fa0cc0a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:24.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fc25cf9c640 1 -- 172.21.15.96:0/4274499220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc25809b5d0 msgr2=0x7fc258130460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fc25cf9c640 1 --2- 172.21.15.96:0/4274499220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc25809b5d0 0x7fc258130460 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fc25cf9c640 1 -- 172.21.15.96:0/4274499220 shutdown_connections 2024-03-21T06:20:24.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fc25cf9c640 1 --2- 172.21.15.96:0/4274499220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc25809b5d0 0x7fc258130460 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fc25cf9c640 1 -- 172.21.15.96:0/4274499220 >> 172.21.15.96:0/4274499220 conn(0x7fc25809fdb0 msgr2=0x7fc2580a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:24.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.799+0000 7fc25cf9c640 1 -- 172.21.15.96:0/4274499220 shutdown_connections 2024-03-21T06:20:24.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.799+0000 7fc25cf9c640 1 -- 172.21.15.96:0/4274499220 wait complete. 2024-03-21T06:20:24.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.799+0000 7f739e264640 1 -- 172.21.15.96:0/3489466914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f739806e820 msgr2=0x7f7398071c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.799+0000 7f739e264640 1 --2- 172.21.15.96:0/3489466914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f739806e820 0x7f7398071c90 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.799+0000 7f739e264640 1 -- 172.21.15.96:0/3489466914 shutdown_connections 2024-03-21T06:20:24.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.799+0000 7fc25cf9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:24.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.799+0000 7f739e264640 1 --2- 172.21.15.96:0/3489466914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f739806e820 0x7f7398071c90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.799+0000 7f739e264640 1 -- 172.21.15.96:0/3489466914 >> 172.21.15.96:0/3489466914 conn(0x7f73980a05c0 msgr2=0x7f73980a09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:24.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.798+0000 7fa0d3cdc640 1 -- 172.21.15.96:0/21656460 shutdown_connections 2024-03-21T06:20:24.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.799+0000 7fa0d3cdc640 1 -- 172.21.15.96:0/21656460 wait complete. 2024-03-21T06:20:24.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.799+0000 7fa0d3cdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:24.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.799+0000 7f739e264640 1 -- 172.21.15.96:0/3489466914 shutdown_connections 2024-03-21T06:20:24.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.799+0000 7f739e264640 1 -- 172.21.15.96:0/3489466914 wait complete. 2024-03-21T06:20:24.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.799+0000 7f739e264640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:24.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.800+0000 7fb3dd15c640 1 Processor -- start 2024-03-21T06:20:24.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.800+0000 7fc25cf9c640 1 Processor -- start 2024-03-21T06:20:24.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.800+0000 7fb3dd15c640 1 -- start start 2024-03-21T06:20:24.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.800+0000 7fa0d3cdc640 1 Processor -- start 2024-03-21T06:20:24.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.800+0000 7fc25cf9c640 1 -- start start 2024-03-21T06:20:24.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.800+0000 7f739e264640 1 Processor -- start 2024-03-21T06:20:24.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.800+0000 7f739e264640 1 -- start start 2024-03-21T06:20:24.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.800+0000 7fa0d3cdc640 1 -- start start 2024-03-21T06:20:24.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.801+0000 7fc25cf9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc25809b5d0 0x7fc258090440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.801+0000 7fc25cf9c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25808ff60 con 0x7fc25809b5d0 2024-03-21T06:20:24.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.801+0000 7fc25659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc25809b5d0 0x7fc258090440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.801+0000 7fc25659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc25809b5d0 0x7fc258090440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35164/0 (socket says 172.21.15.96:35164) 2024-03-21T06:20:24.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.801+0000 7fc25659c640 1 -- 172.21.15.96:0/83450522 learned_addr learned my addr 172.21.15.96:0/83450522 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:24.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.801+0000 7fb3dd15c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d809b650 0x7fb3d811ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.801+0000 7fb3dd15c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d811d770 con 0x7fb3d809b650 2024-03-21T06:20:24.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.801+0000 7f739e264640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f739806e820 0x7f7398133250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.801+0000 7fa0d3cdc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0cc09b360 0x7fa0cc11f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.801+0000 7fa0d3cdc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc122cf0 con 0x7fa0cc09b360 2024-03-21T06:20:24.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.801+0000 7f739e264640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7398135da0 con 0x7f739806e820 2024-03-21T06:20:24.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.801+0000 7fb3d6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d809b650 0x7fb3d811ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.801+0000 7fb3d6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d809b650 0x7fb3d811ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35170/0 (socket says 172.21.15.96:35170) 2024-03-21T06:20:24.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.801+0000 7fb3d6d9d640 1 -- 172.21.15.96:0/470718914 learned_addr learned my addr 172.21.15.96:0/470718914 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:24.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7f7397fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f739806e820 0x7f7398133250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fa0d1a78640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0cc09b360 0x7fa0cc11f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.801+0000 7fc25659c640 1 -- 172.21.15.96:0/83450522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25808d200 con 0x7fc25809b5d0 2024-03-21T06:20:24.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fa0d1a78640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0cc09b360 0x7fa0cc11f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35172/0 (socket says 172.21.15.96:35172) 2024-03-21T06:20:24.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fa0d1a78640 1 -- 172.21.15.96:0/4036948497 learned_addr learned my addr 172.21.15.96:0/4036948497 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:24.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fb3d6d9d640 1 -- 172.21.15.96:0/470718914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d811f700 con 0x7fb3d809b650 2024-03-21T06:20:24.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7f7397fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f739806e820 0x7f7398133250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35176/0 (socket says 172.21.15.96:35176) 2024-03-21T06:20:24.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7f7397fff640 1 -- 172.21.15.96:0/1138638268 learned_addr learned my addr 172.21.15.96:0/1138638268 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:24.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fc25659c640 1 --2- 172.21.15.96:0/83450522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc25809b5d0 0x7fc258090440 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fb3d6d9d640 1 --2- 172.21.15.96:0/470718914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d809b650 0x7fb3d811ded0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fa0d1a78640 1 -- 172.21.15.96:0/4036948497 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc130970 con 0x7fa0cc09b360 2024-03-21T06:20:24.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fb3bf7fe640 1 -- 172.21.15.96:0/470718914 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb3c80058a0 con 0x7fb3d809b650 2024-03-21T06:20:24.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7f7397fff640 1 -- 172.21.15.96:0/1138638268 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7398136cc0 con 0x7f739806e820 2024-03-21T06:20:24.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fc24effd640 1 -- 172.21.15.96:0/83450522 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc248014410 con 0x7fc25809b5d0 2024-03-21T06:20:24.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fb3bf7fe640 1 -- 172.21.15.96:0/470718914 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb3c8005a00 con 0x7fb3d809b650 2024-03-21T06:20:24.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fb3dd15c640 1 -- 172.21.15.96:0/470718914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d8130950 con 0x7fb3d809b650 2024-03-21T06:20:24.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7f7397fff640 1 --2- 172.21.15.96:0/1138638268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f739806e820 0x7f7398133250 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fc24effd640 1 -- 172.21.15.96:0/83450522 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc248014570 con 0x7fc25809b5d0 2024-03-21T06:20:24.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fb3dd15c640 1 -- 172.21.15.96:0/470718914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d8120e20 con 0x7fb3d809b650 2024-03-21T06:20:24.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.803+0000 7fb3bf7fe640 1 -- 172.21.15.96:0/470718914 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb3c8005ca0 con 0x7fb3d809b650 2024-03-21T06:20:24.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.803+0000 7f7394ff9640 1 -- 172.21.15.96:0/1138638268 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f738800e4d0 con 0x7f739806e820 2024-03-21T06:20:24.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.803+0000 7f7394ff9640 1 -- 172.21.15.96:0/1138638268 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f738800ee80 con 0x7f739806e820 2024-03-21T06:20:24.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.803+0000 7f7394ff9640 1 -- 172.21.15.96:0/1138638268 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7388014750 con 0x7f739806e820 2024-03-21T06:20:24.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fa0d1a78640 1 --2- 172.21.15.96:0/4036948497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0cc09b360 0x7fa0cc11f640 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.803+0000 7fb3bd7fa640 1 -- 172.21.15.96:0/470718914 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb3d811d600 con 0x7fb3d809b650 2024-03-21T06:20:24.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.803+0000 7f739e264640 1 -- 172.21.15.96:0/1138638268 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7398136020 con 0x7f739806e820 2024-03-21T06:20:24.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fa0c27fc640 1 -- 172.21.15.96:0/4036948497 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa0c8014540 con 0x7fa0cc09b360 2024-03-21T06:20:24.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fc25cf9c640 1 -- 172.21.15.96:0/83450522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25808f920 con 0x7fc25809b5d0 2024-03-21T06:20:24.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fc25cf9c640 1 -- 172.21.15.96:0/83450522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25808fbc0 con 0x7fc25809b5d0 2024-03-21T06:20:24.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.804+0000 7f739e264640 1 -- 172.21.15.96:0/1138638268 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7398118e40 con 0x7f739806e820 2024-03-21T06:20:24.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.803+0000 7fc25cf9c640 1 -- 172.21.15.96:0/83450522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc25808f570 con 0x7fc25809b5d0 2024-03-21T06:20:24.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.804+0000 7fc24effd640 1 -- 172.21.15.96:0/83450522 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc248014810 con 0x7fc25809b5d0 2024-03-21T06:20:24.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fa0c27fc640 1 -- 172.21.15.96:0/4036948497 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa0c80146a0 con 0x7fa0cc09b360 2024-03-21T06:20:24.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fa0c27fc640 1 -- 172.21.15.96:0/4036948497 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa0c8014940 con 0x7fa0cc09b360 2024-03-21T06:20:24.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fa0d3cdc640 1 -- 172.21.15.96:0/4036948497 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0cc072fc0 con 0x7fa0cc09b360 2024-03-21T06:20:24.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.802+0000 7fa0d3cdc640 1 -- 172.21.15.96:0/4036948497 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0cc123710 con 0x7fa0cc09b360 2024-03-21T06:20:24.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.804+0000 7fa0d3cdc640 1 -- 172.21.15.96:0/4036948497 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa0cc09ba80 con 0x7fa0cc09b360 2024-03-21T06:20:24.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.804+0000 7fb3bf7fe640 1 -- 172.21.15.96:0/470718914 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fb3c80070b0 con 0x7fb3d809b650 2024-03-21T06:20:24.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.804+0000 7fb3bf7fe640 1 --2- 172.21.15.96:0/470718914 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb3c40382b0 0x7fb3c403a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.804+0000 7fb3bf7fe640 1 -- 172.21.15.96:0/470718914 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fb3c8047950 con 0x7fb3d809b650 2024-03-21T06:20:24.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.804+0000 7fa0c27fc640 1 -- 172.21.15.96:0/4036948497 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fa0c8014af0 con 0x7fa0cc09b360 2024-03-21T06:20:24.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.804+0000 7fa0c27fc640 1 --2- 172.21.15.96:0/4036948497 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa0b0038260 0x7fa0b003a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.804+0000 7fa0c27fc640 1 -- 172.21.15.96:0/4036948497 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fa0c8019040 con 0x7fa0cc09b360 2024-03-21T06:20:24.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.804+0000 7fb3bf7fe640 1 --2- 172.21.15.96:0/470718914 >> v2:172.21.15.96:6805/13988474 conn(0x7fb3c403d2b0 0x7fb3c403f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7fb3bf7fe640 1 -- 172.21.15.96:0/470718914 --> v2:172.21.15.96:6805/13988474 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3c403fdd0 con 0x7fb3c403d2b0 2024-03-21T06:20:24.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7fb3bf7fe640 1 -- 172.21.15.96:0/470718914 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (crc 0 0 0) 0x7fb3c8047c90 con 0x7fb3d809b650 2024-03-21T06:20:24.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7fb3d659c640 1 --2- 172.21.15.96:0/470718914 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb3c40382b0 0x7fb3c403a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7fc24effd640 1 -- 172.21.15.96:0/83450522 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fc248014970 con 0x7fc25809b5d0 2024-03-21T06:20:24.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7f7394ff9640 1 -- 172.21.15.96:0/1138638268 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f73880148b0 con 0x7f739806e820 2024-03-21T06:20:24.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7fc24effd640 1 --2- 172.21.15.96:0/83450522 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc2380382b0 0x7fc23803a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7fc24effd640 1 -- 172.21.15.96:0/83450522 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fc248019040 con 0x7fc25809b5d0 2024-03-21T06:20:24.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7f739e264640 1 -- 172.21.15.96:0/1138638268 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f739806aae0 con 0x7f739806e820 2024-03-21T06:20:24.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7fc24effd640 1 --2- 172.21.15.96:0/83450522 >> v2:172.21.15.184:6811/276480837 conn(0x7fc23803d2b0 0x7fc23803f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7fc24effd640 1 -- 172.21.15.96:0/83450522 --> v2:172.21.15.184:6811/276480837 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc23803fdd0 con 0x7fc23803d2b0 2024-03-21T06:20:24.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7fc24effd640 1 -- 172.21.15.96:0/83450522 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (crc 0 0 0) 0x7fc2480493c0 con 0x7fc25809b5d0 2024-03-21T06:20:24.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7f7394ff9640 1 --2- 172.21.15.96:0/1138638268 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f7384038260 0x7f738403a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7f7394ff9640 1 -- 172.21.15.96:0/1138638268 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f7388048f80 con 0x7f739806e820 2024-03-21T06:20:24.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7f7394ff9640 1 --2- 172.21.15.96:0/1138638268 >> v2:172.21.15.184:6804/155456265 conn(0x7f738403d260 0x7f738403f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.806+0000 7f7394ff9640 1 -- 172.21.15.96:0/1138638268 --> v2:172.21.15.184:6804/155456265 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f738403fd80 con 0x7f738403d260 2024-03-21T06:20:24.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.806+0000 7fb3d759e640 1 --2- 172.21.15.96:0/470718914 >> v2:172.21.15.96:6805/13988474 conn(0x7fb3c403d2b0 0x7fb3c403f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:20:24.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.804+0000 7fa0c27fc640 1 --2- 172.21.15.96:0/4036948497 >> v2:172.21.15.96:6801/932229279 conn(0x7fa0b003d260 0x7fa0b003f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:24.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.804+0000 7fa0c27fc640 1 -- 172.21.15.96:0/4036948497 --> v2:172.21.15.96:6801/932229279 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0b003fd80 con 0x7fa0b003d260 2024-03-21T06:20:24.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7fa0d1277640 1 --2- 172.21.15.96:0/4036948497 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa0b0038260 0x7fa0b003a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7fa0c27fc640 1 -- 172.21.15.96:0/4036948497 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (crc 0 0 0) 0x7fa0c8014df0 con 0x7fa0cc09b360 2024-03-21T06:20:24.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7fa0d1277640 1 --2- 172.21.15.96:0/4036948497 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa0b0038260 0x7fa0b003a770 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.805+0000 7fa0d2279640 1 --2- 172.21.15.96:0/4036948497 >> v2:172.21.15.96:6801/932229279 conn(0x7fa0b003d260 0x7fa0b003f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:20:24.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.806+0000 7f73977fe640 1 --2- 172.21.15.96:0/1138638268 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f7384038260 0x7f738403a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.806+0000 7f7394ff9640 1 -- 172.21.15.96:0/1138638268 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (crc 0 0 0) 0x7f738800fba0 con 0x7f739806e820 2024-03-21T06:20:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.806+0000 7fc256d9d640 1 --2- 172.21.15.96:0/83450522 >> v2:172.21.15.184:6811/276480837 conn(0x7fc23803d2b0 0x7fc23803f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:20:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.806+0000 7fc24ffff640 1 --2- 172.21.15.96:0/83450522 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc2380382b0 0x7fc23803a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.806+0000 7fb3d659c640 1 --2- 172.21.15.96:0/470718914 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb3c40382b0 0x7fb3c403a7c0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.806+0000 7f739c801640 1 --2- 172.21.15.96:0/1138638268 >> v2:172.21.15.184:6804/155456265 conn(0x7f738403d260 0x7f738403f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:20:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.807+0000 7f73977fe640 1 --2- 172.21.15.96:0/1138638268 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f7384038260 0x7f738403a770 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.807+0000 7fc24ffff640 1 --2- 172.21.15.96:0/83450522 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc2380382b0 0x7fc23803a7c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.808+0000 7fb3d759e640 1 --2- 172.21.15.96:0/470718914 >> v2:172.21.15.96:6805/13988474 conn(0x7fb3c403d2b0 0x7fb3c403f740 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.808+0000 7fa0d2279640 1 --2- 172.21.15.96:0/4036948497 >> v2:172.21.15.96:6801/932229279 conn(0x7fa0b003d260 0x7fa0b003f6f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.809+0000 7fc256d9d640 1 --2- 172.21.15.96:0/83450522 >> v2:172.21.15.184:6811/276480837 conn(0x7fc23803d2b0 0x7fc23803f740 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.809+0000 7f739c801640 1 --2- 172.21.15.96:0/1138638268 >> v2:172.21.15.184:6804/155456265 conn(0x7f738403d260 0x7f738403f6f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.809+0000 7fa0c27fc640 1 -- 172.21.15.96:0/4036948497 <== osd.0 v2:172.21.15.96:6801/932229279 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7fa0b003fd80 con 0x7fa0b003d260 2024-03-21T06:20:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.809+0000 7fb3bf7fe640 1 -- 172.21.15.96:0/470718914 <== osd.1 v2:172.21.15.96:6805/13988474 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7fb3c403fdd0 con 0x7fb3c403d2b0 2024-03-21T06:20:24.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.810+0000 7fc24effd640 1 -- 172.21.15.96:0/83450522 <== osd.3 v2:172.21.15.184:6811/276480837 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7fc23803fdd0 con 0x7fc23803d2b0 2024-03-21T06:20:24.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.810+0000 7f7394ff9640 1 -- 172.21.15.96:0/1138638268 <== osd.2 v2:172.21.15.184:6804/155456265 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7f738403fd80 con 0x7f738403d260 2024-03-21T06:20:24.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.813+0000 7fa0d3cdc640 1 -- 172.21.15.96:0/4036948497 --> v2:172.21.15.96:6801/932229279 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa0cc121bb0 con 0x7fa0b003d260 2024-03-21T06:20:24.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.814+0000 7fa0c27fc640 1 -- 172.21.15.96:0/4036948497 <== osd.0 v2:172.21.15.96:6801/932229279 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa0cc121bb0 con 0x7fa0b003d260 2024-03-21T06:20:24.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.814+0000 7f739e264640 1 -- 172.21.15.96:0/1138638268 --> v2:172.21.15.184:6804/155456265 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7398137c50 con 0x7f738403d260 2024-03-21T06:20:24.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.814+0000 7fa0d3cdc640 1 -- 172.21.15.96:0/4036948497 >> v2:172.21.15.96:6801/932229279 conn(0x7fa0b003d260 msgr2=0x7fa0b003f6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.814+0000 7fa0d3cdc640 1 --2- 172.21.15.96:0/4036948497 >> v2:172.21.15.96:6801/932229279 conn(0x7fa0b003d260 0x7fa0b003f6f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.815+0000 7fa0d3cdc640 1 -- 172.21.15.96:0/4036948497 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa0b0038260 msgr2=0x7fa0b003a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.815+0000 7fa0d3cdc640 1 --2- 172.21.15.96:0/4036948497 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa0b0038260 0x7fa0b003a770 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.815+0000 7fa0d3cdc640 1 -- 172.21.15.96:0/4036948497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0cc09b360 msgr2=0x7fa0cc11f640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.815+0000 7fa0d3cdc640 1 --2- 172.21.15.96:0/4036948497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0cc09b360 0x7fa0cc11f640 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.815+0000 7fa0d3cdc640 1 -- 172.21.15.96:0/4036948497 shutdown_connections 2024-03-21T06:20:24.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.815+0000 7fa0d3cdc640 1 --2- 172.21.15.96:0/4036948497 >> v2:172.21.15.96:6801/932229279 conn(0x7fa0b003d260 0x7fa0b003f6f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.815+0000 7fa0d3cdc640 1 --2- 172.21.15.96:0/4036948497 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa0b0038260 0x7fa0b003a770 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.815+0000 7fa0d3cdc640 1 --2- 172.21.15.96:0/4036948497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0cc09b360 0x7fa0cc11f640 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.815+0000 7fa0d3cdc640 1 -- 172.21.15.96:0/4036948497 >> 172.21.15.96:0/4036948497 conn(0x7fa0cc0a42c0 msgr2=0x7fa0cc09f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:24.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.815+0000 7fa0d3cdc640 1 -- 172.21.15.96:0/4036948497 shutdown_connections 2024-03-21T06:20:24.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.815+0000 7fa0d3cdc640 1 -- 172.21.15.96:0/4036948497 wait complete. 2024-03-21T06:20:24.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.816+0000 7f7394ff9640 1 -- 172.21.15.96:0/1138638268 <== osd.2 v2:172.21.15.184:6804/155456265 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7398137c50 con 0x7f738403d260 2024-03-21T06:20:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.816+0000 7f739e264640 1 -- 172.21.15.96:0/1138638268 >> v2:172.21.15.184:6804/155456265 conn(0x7f738403d260 msgr2=0x7f738403f6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.816+0000 7f739e264640 1 --2- 172.21.15.96:0/1138638268 >> v2:172.21.15.184:6804/155456265 conn(0x7f738403d260 0x7f738403f6f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.816+0000 7f739e264640 1 -- 172.21.15.96:0/1138638268 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f7384038260 msgr2=0x7f738403a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.816+0000 7f739e264640 1 --2- 172.21.15.96:0/1138638268 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f7384038260 0x7f738403a770 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.816+0000 7f739e264640 1 -- 172.21.15.96:0/1138638268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f739806e820 msgr2=0x7f7398133250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.816+0000 7f739e264640 1 --2- 172.21.15.96:0/1138638268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f739806e820 0x7f7398133250 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.816+0000 7f739e264640 1 -- 172.21.15.96:0/1138638268 shutdown_connections 2024-03-21T06:20:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.816+0000 7f739e264640 1 --2- 172.21.15.96:0/1138638268 >> v2:172.21.15.184:6804/155456265 conn(0x7f738403d260 0x7f738403f6f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.816+0000 7f739e264640 1 --2- 172.21.15.96:0/1138638268 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f7384038260 0x7f738403a770 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.816+0000 7f739e264640 1 --2- 172.21.15.96:0/1138638268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f739806e820 0x7f7398133250 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.816+0000 7f739e264640 1 -- 172.21.15.96:0/1138638268 >> 172.21.15.96:0/1138638268 conn(0x7f73980a05c0 msgr2=0x7f739809d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.816+0000 7f739e264640 1 -- 172.21.15.96:0/1138638268 shutdown_connections 2024-03-21T06:20:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.816+0000 7f739e264640 1 -- 172.21.15.96:0/1138638268 wait complete. 2024-03-21T06:20:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.817+0000 7fb3bd7fa640 1 -- 172.21.15.96:0/470718914 --> v2:172.21.15.96:6805/13988474 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb3d809bd70 con 0x7fb3c403d2b0 2024-03-21T06:20:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.817+0000 7fc25cf9c640 1 -- 172.21.15.96:0/83450522 --> v2:172.21.15.184:6811/276480837 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc258062f10 con 0x7fc23803d2b0 2024-03-21T06:20:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.818+0000 7fb3bf7fe640 1 -- 172.21.15.96:0/470718914 <== osd.1 v2:172.21.15.96:6805/13988474 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb3d000a820 con 0x7fb3c403d2b0 2024-03-21T06:20:24.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.818+0000 7fc24effd640 1 -- 172.21.15.96:0/83450522 <== osd.3 v2:172.21.15.184:6811/276480837 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc258062f10 con 0x7fc23803d2b0 2024-03-21T06:20:24.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.818+0000 7fb3bd7fa640 1 -- 172.21.15.96:0/470718914 >> v2:172.21.15.96:6805/13988474 conn(0x7fb3c403d2b0 msgr2=0x7fb3c403f740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.818+0000 7fb3bd7fa640 1 --2- 172.21.15.96:0/470718914 >> v2:172.21.15.96:6805/13988474 conn(0x7fb3c403d2b0 0x7fb3c403f740 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.818+0000 7fc25cf9c640 1 -- 172.21.15.96:0/83450522 >> v2:172.21.15.184:6811/276480837 conn(0x7fc23803d2b0 msgr2=0x7fc23803f740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.818+0000 7fc25cf9c640 1 --2- 172.21.15.96:0/83450522 >> v2:172.21.15.184:6811/276480837 conn(0x7fc23803d2b0 0x7fc23803f740 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.818+0000 7fb3bd7fa640 1 -- 172.21.15.96:0/470718914 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb3c40382b0 msgr2=0x7fb3c403a7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.818+0000 7fb3bd7fa640 1 --2- 172.21.15.96:0/470718914 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb3c40382b0 0x7fb3c403a7c0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.818+0000 7fb3bd7fa640 1 -- 172.21.15.96:0/470718914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d809b650 msgr2=0x7fb3d811ded0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.818+0000 7fb3bd7fa640 1 --2- 172.21.15.96:0/470718914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d809b650 0x7fb3d811ded0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fb3bd7fa640 1 -- 172.21.15.96:0/470718914 shutdown_connections 2024-03-21T06:20:24.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fb3bd7fa640 1 --2- 172.21.15.96:0/470718914 >> v2:172.21.15.96:6805/13988474 conn(0x7fb3c403d2b0 0x7fb3c403f740 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fb3bd7fa640 1 --2- 172.21.15.96:0/470718914 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb3c40382b0 0x7fb3c403a7c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fb3bd7fa640 1 --2- 172.21.15.96:0/470718914 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3d809b650 0x7fb3d811ded0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fb3bd7fa640 1 -- 172.21.15.96:0/470718914 >> 172.21.15.96:0/470718914 conn(0x7fb3d80a45c0 msgr2=0x7fb3d809d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:24.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fc25cf9c640 1 -- 172.21.15.96:0/83450522 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc2380382b0 msgr2=0x7fc23803a7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fc25cf9c640 1 --2- 172.21.15.96:0/83450522 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc2380382b0 0x7fc23803a7c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fc25cf9c640 1 -- 172.21.15.96:0/83450522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc25809b5d0 msgr2=0x7fc258090440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:24.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fc25cf9c640 1 --2- 172.21.15.96:0/83450522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc25809b5d0 0x7fc258090440 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fc25cf9c640 1 -- 172.21.15.96:0/83450522 shutdown_connections 2024-03-21T06:20:24.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fc25cf9c640 1 --2- 172.21.15.96:0/83450522 >> v2:172.21.15.184:6811/276480837 conn(0x7fc23803d2b0 0x7fc23803f740 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fc25cf9c640 1 --2- 172.21.15.96:0/83450522 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc2380382b0 0x7fc23803a7c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fc25cf9c640 1 --2- 172.21.15.96:0/83450522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc25809b5d0 0x7fc258090440 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:24.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fc25cf9c640 1 -- 172.21.15.96:0/83450522 >> 172.21.15.96:0/83450522 conn(0x7fc25809fdb0 msgr2=0x7fc2580a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:24.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fb3bd7fa640 1 -- 172.21.15.96:0/470718914 shutdown_connections 2024-03-21T06:20:24.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fb3bd7fa640 1 -- 172.21.15.96:0/470718914 wait complete. 2024-03-21T06:20:24.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fc25cf9c640 1 -- 172.21.15.96:0/83450522 shutdown_connections 2024-03-21T06:20:24.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:24.819+0000 7fc25cf9c640 1 -- 172.21.15.96:0/83450522 wait complete. 2024-03-21T06:20:24.853 INFO:teuthology.orchestra.run.smithi096.stdout:38654705670 2024-03-21T06:20:24.854 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-21T06:20:24.855 INFO:teuthology.orchestra.run.smithi096.stdout:38654705670 2024-03-21T06:20:24.855 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-21T06:20:24.856 INFO:teuthology.orchestra.run.smithi096.stdout:38654705670 2024-03-21T06:20:24.856 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-21T06:20:24.857 INFO:teuthology.orchestra.run.smithi096.stdout:38654705670 2024-03-21T06:20:24.857 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-21T06:20:25.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.006+0000 7f32bad5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:25.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.006+0000 7fcc8ae64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:25.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.006+0000 7fdb1acdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:25.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.006+0000 7fb279824640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:25.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.011+0000 7f32bad5c640 1 Processor -- start 2024-03-21T06:20:25.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.011+0000 7fdb1acdc640 1 Processor -- start 2024-03-21T06:20:25.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.011+0000 7fb279824640 1 Processor -- start 2024-03-21T06:20:25.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.011+0000 7fdb1acdc640 1 -- start start 2024-03-21T06:20:25.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.011+0000 7fb279824640 1 -- start start 2024-03-21T06:20:25.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.012+0000 7fcc8ae64640 1 Processor -- start 2024-03-21T06:20:25.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.012+0000 7fcc8ae64640 1 -- start start 2024-03-21T06:20:25.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.011+0000 7f32bad5c640 1 -- start start 2024-03-21T06:20:25.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.012+0000 7f32bad5c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f32b409d7c0 0x7f32b412ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.012+0000 7f32bad5c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b40a09e0 con 0x7f32b412f460 2024-03-21T06:20:25.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.012+0000 7f32bad5c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b409cc90 con 0x7f32b409d7c0 2024-03-21T06:20:25.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.012+0000 7f32b8af8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f32b409d7c0 0x7f32b412ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.012+0000 7f32b3fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f32b412f460 0x7f32b4131e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33154/0 (socket says 172.21.15.96:33154) 2024-03-21T06:20:25.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.012+0000 7f32b3fff640 1 -- 172.21.15.96:0/1662050231 learned_addr learned my addr 172.21.15.96:0/1662050231 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:25.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.012+0000 7fcc8ae64640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcc8409b5d0 0x7fcc84130460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.012+0000 7fcc8ae64640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc84099dc0 con 0x7fcc841309a0 2024-03-21T06:20:25.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.012+0000 7fcc8ae64640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc8409bae0 con 0x7fcc8409b5d0 2024-03-21T06:20:25.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fcc7bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fcc841309a0 0x7fcc84098190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33156/0 (socket says 172.21.15.96:33156) 2024-03-21T06:20:25.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fcc7bfff640 1 -- 172.21.15.96:0/1973711890 learned_addr learned my addr 172.21.15.96:0/1973711890 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:25.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fb279824640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb27409b560 0x7fb2741398a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7f32b8af8640 1 -- 172.21.15.96:0/1662050231 >> v1:172.21.15.96:6789/0 conn(0x7f32b412f460 legacy=0x7f32b4131e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7f32b8af8640 1 -- 172.21.15.96:0/1662050231 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b409dc90 con 0x7f32b409d7c0 2024-03-21T06:20:25.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fb279824640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb27406e370 con 0x7fb27409aa40 2024-03-21T06:20:25.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fcc88c00640 1 --2- 172.21.15.96:0/1973711890 >> v2:172.21.15.96:3300/0 conn(0x7fcc8409b5d0 0x7fcc84130460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fb279824640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb27406dd50 con 0x7fb27409b560 2024-03-21T06:20:25.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fb272ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fb27409aa40 0x7fb27409ae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33158/0 (socket says 172.21.15.96:33158) 2024-03-21T06:20:25.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fb272ffd640 1 -- 172.21.15.96:0/378044478 learned_addr learned my addr 172.21.15.96:0/378044478 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:25.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fb2727fc640 1 --2- 172.21.15.96:0/378044478 >> v2:172.21.15.96:3300/0 conn(0x7fb27409b560 0x7fb2741398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7f32b8af8640 1 --2- 172.21.15.96:0/1662050231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32b409d7c0 0x7f32b412ef20 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10769896f25db330 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fcc7b7fe640 1 -- 172.21.15.96:0/1973711890 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3819456383 0 0) 0x7fcc84099dc0 con 0x7fcc841309a0 2024-03-21T06:20:25.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fcc7b7fe640 1 -- 172.21.15.96:0/1973711890 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc6c003410 con 0x7fcc841309a0 2024-03-21T06:20:25.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fdb1acdc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdb14098e20 0x7fdb14135780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fdb1acdc640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb14099f50 con 0x7fdb1409b5f0 2024-03-21T06:20:25.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fdb1acdc640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb14099c40 con 0x7fdb14098e20 2024-03-21T06:20:25.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7f32b37fe640 1 -- 172.21.15.96:0/1662050231 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f32a8009ce0 con 0x7f32b409d7c0 2024-03-21T06:20:25.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fdb13fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdb14098e20 0x7fdb14135780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fcc88c00640 1 -- 172.21.15.96:0/1973711890 >> v1:172.21.15.96:6789/0 conn(0x7fcc841309a0 legacy=0x7fcc84098190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fdb18a78640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fdb1409b5f0 0x7fdb14098710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33166/0 (socket says 172.21.15.96:33166) 2024-03-21T06:20:25.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fdb13fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdb14098e20 0x7fdb14135780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35204/0 (socket says 172.21.15.96:35204) 2024-03-21T06:20:25.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7f32b37fe640 1 -- 172.21.15.96:0/1662050231 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f32a8009e40 con 0x7f32b409d7c0 2024-03-21T06:20:25.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7f32b37fe640 1 -- 172.21.15.96:0/1662050231 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f32a800e360 con 0x7f32b409d7c0 2024-03-21T06:20:25.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fcc88c00640 1 -- 172.21.15.96:0/1973711890 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8409c0f0 con 0x7fcc8409b5d0 2024-03-21T06:20:25.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fdb13fff640 1 -- 172.21.15.96:0/1407312468 learned_addr learned my addr 172.21.15.96:0/1407312468 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:25.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fb271ffb640 1 -- 172.21.15.96:0/378044478 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98861318 0 0) 0x7fb27406e370 con 0x7fb27409aa40 2024-03-21T06:20:25.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fb271ffb640 1 -- 172.21.15.96:0/378044478 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb25c0034f0 con 0x7fb27409aa40 2024-03-21T06:20:25.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.013+0000 7fcc88c00640 1 --2- 172.21.15.96:0/1973711890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc8409b5d0 0x7fcc84130460 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d68beed9a4d6b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.014+0000 7fb2727fc640 1 -- 172.21.15.96:0/378044478 >> v1:172.21.15.96:6789/0 conn(0x7fb27409aa40 legacy=0x7fb27409ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.014+0000 7fb2727fc640 1 -- 172.21.15.96:0/378044478 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27409cbd0 con 0x7fb27409b560 2024-03-21T06:20:25.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.014+0000 7fcc7b7fe640 1 -- 172.21.15.96:0/1973711890 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcc80009ce0 con 0x7fcc8409b5d0 2024-03-21T06:20:25.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.014+0000 7fdb13fff640 1 -- 172.21.15.96:0/1407312468 >> v1:172.21.15.96:6789/0 conn(0x7fdb1409b5f0 legacy=0x7fdb14098710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.014+0000 7fdb13fff640 1 -- 172.21.15.96:0/1407312468 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1406da00 con 0x7fdb14098e20 2024-03-21T06:20:25.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.014+0000 7fcc7b7fe640 1 -- 172.21.15.96:0/1973711890 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fcc80009e40 con 0x7fcc8409b5d0 2024-03-21T06:20:25.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.014+0000 7fcc7b7fe640 1 -- 172.21.15.96:0/1973711890 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcc8000e360 con 0x7fcc8409b5d0 2024-03-21T06:20:25.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.014+0000 7fb2727fc640 1 --2- 172.21.15.96:0/378044478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27409b560 0x7fb2741398a0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec2bcefcafb37a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.014+0000 7fdb13fff640 1 --2- 172.21.15.96:0/1407312468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb14098e20 0x7fdb14135780 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d985b1424cdbfc server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.014+0000 7fb271ffb640 1 -- 172.21.15.96:0/378044478 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb268009ce0 con 0x7fb27409b560 2024-03-21T06:20:25.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.014+0000 7fb271ffb640 1 -- 172.21.15.96:0/378044478 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb268009e40 con 0x7fb27409b560 2024-03-21T06:20:25.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.014+0000 7fb271ffb640 1 -- 172.21.15.96:0/378044478 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb26800e360 con 0x7fb27409b560 2024-03-21T06:20:25.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.014+0000 7fdb137fe640 1 -- 172.21.15.96:0/1407312468 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdb00009dd0 con 0x7fdb14098e20 2024-03-21T06:20:25.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.014+0000 7fdb137fe640 1 -- 172.21.15.96:0/1407312468 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fdb000103b0 con 0x7fdb14098e20 2024-03-21T06:20:25.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.014+0000 7fdb137fe640 1 -- 172.21.15.96:0/1407312468 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdb00010650 con 0x7fdb14098e20 2024-03-21T06:20:25.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.014+0000 7f32bad5c640 1 -- 172.21.15.96:0/1662050231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32b409d7c0 msgr2=0x7f32b412ef20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.014+0000 7f32bad5c640 1 --2- 172.21.15.96:0/1662050231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32b409d7c0 0x7f32b412ef20 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7f32bad5c640 1 -- 172.21.15.96:0/1662050231 shutdown_connections 2024-03-21T06:20:25.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7f32bad5c640 1 --2- 172.21.15.96:0/1662050231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32b409d7c0 0x7f32b412ef20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7f32bad5c640 1 -- 172.21.15.96:0/1662050231 >> 172.21.15.96:0/1662050231 conn(0x7f32b40a0da0 msgr2=0x7f32b40a11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:25.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7f32bad5c640 1 -- 172.21.15.96:0/1662050231 shutdown_connections 2024-03-21T06:20:25.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7f32bad5c640 1 -- 172.21.15.96:0/1662050231 wait complete. 2024-03-21T06:20:25.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fcc8ae64640 1 -- 172.21.15.96:0/1973711890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc8409b5d0 msgr2=0x7fcc84130460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fcc8ae64640 1 --2- 172.21.15.96:0/1973711890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc8409b5d0 0x7fcc84130460 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fcc8ae64640 1 -- 172.21.15.96:0/1973711890 shutdown_connections 2024-03-21T06:20:25.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fb279824640 1 -- 172.21.15.96:0/378044478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27409b560 msgr2=0x7fb2741398a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fb279824640 1 --2- 172.21.15.96:0/378044478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27409b560 0x7fb2741398a0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fcc8ae64640 1 --2- 172.21.15.96:0/1973711890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc8409b5d0 0x7fcc84130460 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7f32bad5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:25.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fb279824640 1 -- 172.21.15.96:0/378044478 shutdown_connections 2024-03-21T06:20:25.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fb279824640 1 --2- 172.21.15.96:0/378044478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27409b560 0x7fb2741398a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fb279824640 1 -- 172.21.15.96:0/378044478 >> 172.21.15.96:0/378044478 conn(0x7fb27409d2f0 msgr2=0x7fb27409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:25.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fcc8ae64640 1 -- 172.21.15.96:0/1973711890 >> 172.21.15.96:0/1973711890 conn(0x7fcc8409fdb0 msgr2=0x7fcc840a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:25.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fdb1acdc640 1 -- 172.21.15.96:0/1407312468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb14098e20 msgr2=0x7fdb14135780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fcc8ae64640 1 -- 172.21.15.96:0/1973711890 shutdown_connections 2024-03-21T06:20:25.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fdb1acdc640 1 --2- 172.21.15.96:0/1407312468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb14098e20 0x7fdb14135780 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fcc8ae64640 1 -- 172.21.15.96:0/1973711890 wait complete. 2024-03-21T06:20:25.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fb279824640 1 -- 172.21.15.96:0/378044478 shutdown_connections 2024-03-21T06:20:25.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fb279824640 1 -- 172.21.15.96:0/378044478 wait complete. 2024-03-21T06:20:25.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fdb1acdc640 1 -- 172.21.15.96:0/1407312468 shutdown_connections 2024-03-21T06:20:25.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fdb1acdc640 1 --2- 172.21.15.96:0/1407312468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb14098e20 0x7fdb14135780 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fdb1acdc640 1 -- 172.21.15.96:0/1407312468 >> 172.21.15.96:0/1407312468 conn(0x7fdb1409fdb0 msgr2=0x7fdb140a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:25.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fcc8ae64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:25.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fdb1acdc640 1 -- 172.21.15.96:0/1407312468 shutdown_connections 2024-03-21T06:20:25.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fdb1acdc640 1 -- 172.21.15.96:0/1407312468 wait complete. 2024-03-21T06:20:25.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.015+0000 7fb279824640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:25.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.016+0000 7fdb1acdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:25.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.016+0000 7f32bad5c640 1 Processor -- start 2024-03-21T06:20:25.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.016+0000 7f32bad5c640 1 -- start start 2024-03-21T06:20:25.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.016+0000 7fb279824640 1 Processor -- start 2024-03-21T06:20:25.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.016+0000 7fcc8ae64640 1 Processor -- start 2024-03-21T06:20:25.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.016+0000 7fdb1acdc640 1 Processor -- start 2024-03-21T06:20:25.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.016+0000 7fcc8ae64640 1 -- start start 2024-03-21T06:20:25.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.017+0000 7fdb1acdc640 1 -- start start 2024-03-21T06:20:25.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.017+0000 7fb279824640 1 -- start start 2024-03-21T06:20:25.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.017+0000 7f32bad5c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32b409d7c0 0x7f32b411a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.017+0000 7f32bad5c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b411e4c0 con 0x7f32b409d7c0 2024-03-21T06:20:25.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.017+0000 7f32b8af8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32b409d7c0 0x7f32b411a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.017+0000 7f32b8af8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32b409d7c0 0x7f32b411a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35210/0 (socket says 172.21.15.96:35210) 2024-03-21T06:20:25.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.017+0000 7f32b8af8640 1 -- 172.21.15.96:0/321452714 learned_addr learned my addr 172.21.15.96:0/321452714 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:25.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.017+0000 7fb279824640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27409aa40 0x7fb274123f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.017+0000 7fb279824640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2741352d0 con 0x7fb27409aa40 2024-03-21T06:20:25.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.017+0000 7fdb1acdc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb14098e20 0x7fdb1412db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.017+0000 7fdb1acdc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1412bc70 con 0x7fdb14098e20 2024-03-21T06:20:25.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.017+0000 7fcc8ae64640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc8409b5d0 0x7fcc84128010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.017+0000 7fcc8ae64640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc84129e30 con 0x7fcc8409b5d0 2024-03-21T06:20:25.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.017+0000 7fb272ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27409aa40 0x7fb274123f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.017+0000 7f32b8af8640 1 -- 172.21.15.96:0/321452714 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b406b6e0 con 0x7f32b409d7c0 2024-03-21T06:20:25.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.017+0000 7fb272ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27409aa40 0x7fb274123f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35224/0 (socket says 172.21.15.96:35224) 2024-03-21T06:20:25.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.017+0000 7fb272ffd640 1 -- 172.21.15.96:0/2296972601 learned_addr learned my addr 172.21.15.96:0/2296972601 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:25.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7fdb18a78640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb14098e20 0x7fdb1412db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7fcc88c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc8409b5d0 0x7fcc84128010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7fdb18a78640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb14098e20 0x7fdb1412db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35240/0 (socket says 172.21.15.96:35240) 2024-03-21T06:20:25.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7fdb18a78640 1 -- 172.21.15.96:0/2745762842 learned_addr learned my addr 172.21.15.96:0/2745762842 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:25.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7fcc88c00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc8409b5d0 0x7fcc84128010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35254/0 (socket says 172.21.15.96:35254) 2024-03-21T06:20:25.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7fcc88c00640 1 -- 172.21.15.96:0/2115579504 learned_addr learned my addr 172.21.15.96:0/2115579504 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:25.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7f32b8af8640 1 --2- 172.21.15.96:0/321452714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32b409d7c0 0x7f32b411a750 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7fb272ffd640 1 -- 172.21.15.96:0/2296972601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27406c0f0 con 0x7fb27409aa40 2024-03-21T06:20:25.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7fb272ffd640 1 --2- 172.21.15.96:0/2296972601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27409aa40 0x7fb274123f60 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7f32b17fa640 1 -- 172.21.15.96:0/321452714 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f32a8014620 con 0x7f32b409d7c0 2024-03-21T06:20:25.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7f32bad5c640 1 -- 172.21.15.96:0/321452714 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b411d9b0 con 0x7f32b409d7c0 2024-03-21T06:20:25.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7fb25bfff640 1 -- 172.21.15.96:0/2296972601 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb2640059d0 con 0x7fb27409aa40 2024-03-21T06:20:25.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7fb25bfff640 1 -- 172.21.15.96:0/2296972601 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb264005b30 con 0x7fb27409aa40 2024-03-21T06:20:25.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7fcc88c00640 1 -- 172.21.15.96:0/2115579504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc84127b50 con 0x7fcc8409b5d0 2024-03-21T06:20:25.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7f32b17fa640 1 -- 172.21.15.96:0/321452714 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f32a8014780 con 0x7f32b409d7c0 2024-03-21T06:20:25.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7f32b17fa640 1 -- 172.21.15.96:0/321452714 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f32a8014a20 con 0x7f32b409d7c0 2024-03-21T06:20:25.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7fb25bfff640 1 -- 172.21.15.96:0/2296972601 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb264005dd0 con 0x7fb27409aa40 2024-03-21T06:20:25.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7f32bad5c640 1 -- 172.21.15.96:0/321452714 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b411bf60 con 0x7f32b409d7c0 2024-03-21T06:20:25.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7fdb18a78640 1 -- 172.21.15.96:0/2745762842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb14073990 con 0x7fdb14098e20 2024-03-21T06:20:25.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7fcc88c00640 1 --2- 172.21.15.96:0/2115579504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc8409b5d0 0x7fcc84128010 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.018+0000 7fb279824640 1 -- 172.21.15.96:0/2296972601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb274134fe0 con 0x7fb27409aa40 2024-03-21T06:20:25.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.019+0000 7fb279824640 1 -- 172.21.15.96:0/2296972601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb274125b80 con 0x7fb27409aa40 2024-03-21T06:20:25.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.019+0000 7fcc797fa640 1 -- 172.21.15.96:0/2115579504 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcc8000e360 con 0x7fcc8409b5d0 2024-03-21T06:20:25.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.019+0000 7fcc797fa640 1 -- 172.21.15.96:0/2115579504 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fcc80009c90 con 0x7fcc8409b5d0 2024-03-21T06:20:25.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.019+0000 7fcc8ae64640 1 -- 172.21.15.96:0/2115579504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc84127840 con 0x7fcc8409b5d0 2024-03-21T06:20:25.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.019+0000 7fcc8ae64640 1 -- 172.21.15.96:0/2115579504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8412b2f0 con 0x7fcc8409b5d0 2024-03-21T06:20:25.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.020+0000 7f32bad5c640 1 -- 172.21.15.96:0/321452714 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b4062f10 con 0x7f32b409d7c0 2024-03-21T06:20:25.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.020+0000 7fdb18a78640 1 --2- 172.21.15.96:0/2745762842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb14098e20 0x7fdb1412db10 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.020+0000 7fcc797fa640 1 -- 172.21.15.96:0/2115579504 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcc80014750 con 0x7fcc8409b5d0 2024-03-21T06:20:25.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.019+0000 7fb25bfff640 1 -- 172.21.15.96:0/2296972601 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fb2640070b0 con 0x7fb27409aa40 2024-03-21T06:20:25.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.020+0000 7fb25bfff640 1 --2- 172.21.15.96:0/2296972601 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb260038210 0x7fb26003a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.020+0000 7fb25bfff640 1 -- 172.21.15.96:0/2296972601 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fb2640479b0 con 0x7fb27409aa40 2024-03-21T06:20:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.020+0000 7f32b17fa640 1 -- 172.21.15.96:0/321452714 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f32a8014b80 con 0x7f32b409d7c0 2024-03-21T06:20:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.020+0000 7f32b17fa640 1 --2- 172.21.15.96:0/321452714 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f32980382b0 0x7f329803a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.020+0000 7f32b17fa640 1 -- 172.21.15.96:0/321452714 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f32a8019040 con 0x7f32b409d7c0 2024-03-21T06:20:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.020+0000 7fb259ffb640 1 -- 172.21.15.96:0/2296972601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb24c005050 con 0x7fb27409aa40 2024-03-21T06:20:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.020+0000 7fb2727fc640 1 --2- 172.21.15.96:0/2296972601 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb260038210 0x7fb26003a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.020+0000 7f32b3fff640 1 --2- 172.21.15.96:0/321452714 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f32980382b0 0x7f329803a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.020+0000 7fcc797fa640 1 -- 172.21.15.96:0/2115579504 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fcc800148b0 con 0x7fcc8409b5d0 2024-03-21T06:20:25.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.021+0000 7fb2727fc640 1 --2- 172.21.15.96:0/2296972601 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb260038210 0x7fb26003a720 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.021+0000 7fcc797fa640 1 --2- 172.21.15.96:0/2115579504 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcc70038300 0x7fcc7003a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.021+0000 7fcc797fa640 1 -- 172.21.15.96:0/2115579504 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fcc80019040 con 0x7fcc8409b5d0 2024-03-21T06:20:25.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.021+0000 7fcc8ae64640 1 -- 172.21.15.96:0/2115579504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc84062f10 con 0x7fcc8409b5d0 2024-03-21T06:20:25.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.021+0000 7fcc7bfff640 1 --2- 172.21.15.96:0/2115579504 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcc70038300 0x7fcc7003a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.021+0000 7f32b3fff640 1 --2- 172.21.15.96:0/321452714 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f32980382b0 0x7f329803a7c0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.021+0000 7fcc7bfff640 1 --2- 172.21.15.96:0/2115579504 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcc70038300 0x7fcc7003a810 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.025+0000 7f32b17fa640 1 -- 172.21.15.96:0/321452714 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f32a8018c50 con 0x7f32b409d7c0 2024-03-21T06:20:25.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.029+0000 7fdb117fa640 1 -- 172.21.15.96:0/2745762842 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdb080071e0 con 0x7fdb14098e20 2024-03-21T06:20:25.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.030+0000 7fdb1acdc640 1 -- 172.21.15.96:0/2745762842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb1412afd0 con 0x7fdb14098e20 2024-03-21T06:20:25.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.030+0000 7fb25bfff640 1 -- 172.21.15.96:0/2296972601 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fb26400da20 con 0x7fb27409aa40 2024-03-21T06:20:25.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.030+0000 7fdb1acdc640 1 -- 172.21.15.96:0/2745762842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1412b590 con 0x7fdb14098e20 2024-03-21T06:20:25.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.031+0000 7fdb1acdc640 1 -- 172.21.15.96:0/2745762842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb14062f10 con 0x7fdb14098e20 2024-03-21T06:20:25.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.039+0000 7fdb117fa640 1 -- 172.21.15.96:0/2745762842 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fdb08007340 con 0x7fdb14098e20 2024-03-21T06:20:25.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.039+0000 7fdb117fa640 1 -- 172.21.15.96:0/2745762842 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fdb080075e0 con 0x7fdb14098e20 2024-03-21T06:20:25.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.039+0000 7fcc797fa640 1 -- 172.21.15.96:0/2115579504 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fcc8000f990 con 0x7fcc8409b5d0 2024-03-21T06:20:25.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.040+0000 7fdb117fa640 1 -- 172.21.15.96:0/2745762842 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fdb080057e0 con 0x7fdb14098e20 2024-03-21T06:20:25.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.040+0000 7fdb117fa640 1 --2- 172.21.15.96:0/2745762842 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fdaf80382b0 0x7fdaf803a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.040+0000 7fdb117fa640 1 -- 172.21.15.96:0/2745762842 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fdb080479f0 con 0x7fdb14098e20 2024-03-21T06:20:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.040+0000 7fdb13fff640 1 --2- 172.21.15.96:0/2745762842 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fdaf80382b0 0x7fdaf803a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.041+0000 7fdb13fff640 1 --2- 172.21.15.96:0/2745762842 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fdaf80382b0 0x7fdaf803a7c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.049+0000 7fdb117fa640 1 -- 172.21.15.96:0/2745762842 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fdb0800d740 con 0x7fdb14098e20 2024-03-21T06:20:25.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.209+0000 7f32bad5c640 1 -- 172.21.15.96:0/321452714 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f32b406b6e0 con 0x7f32b409d7c0 2024-03-21T06:20:25.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.209+0000 7f32b17fa640 1 -- 172.21.15.96:0/321452714 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (crc 0 0 0) 0x7f32b406b6e0 con 0x7f32b409d7c0 2024-03-21T06:20:25.210 INFO:teuthology.orchestra.run.smithi096.stdout:38654705670 2024-03-21T06:20:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.211+0000 7f32bad5c640 1 -- 172.21.15.96:0/321452714 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f32980382b0 msgr2=0x7f329803a7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.211+0000 7f32bad5c640 1 --2- 172.21.15.96:0/321452714 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f32980382b0 0x7f329803a7c0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.211+0000 7f32bad5c640 1 -- 172.21.15.96:0/321452714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32b409d7c0 msgr2=0x7f32b411a750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.211+0000 7f32bad5c640 1 --2- 172.21.15.96:0/321452714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32b409d7c0 0x7f32b411a750 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.211+0000 7f32bad5c640 1 -- 172.21.15.96:0/321452714 shutdown_connections 2024-03-21T06:20:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.211+0000 7f32bad5c640 1 --2- 172.21.15.96:0/321452714 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f32980382b0 0x7f329803a7c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.211+0000 7f32bad5c640 1 --2- 172.21.15.96:0/321452714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32b409d7c0 0x7f32b411a750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.211+0000 7f32bad5c640 1 -- 172.21.15.96:0/321452714 >> 172.21.15.96:0/321452714 conn(0x7f32b40a0da0 msgr2=0x7f32b41ba0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:25.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.211+0000 7f32bad5c640 1 -- 172.21.15.96:0/321452714 shutdown_connections 2024-03-21T06:20:25.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.212+0000 7f32bad5c640 1 -- 172.21.15.96:0/321452714 wait complete. 2024-03-21T06:20:25.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.221+0000 7fcc8ae64640 1 -- 172.21.15.96:0/2115579504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcc84074b30 con 0x7fcc8409b5d0 2024-03-21T06:20:25.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.222+0000 7fcc797fa640 1 -- 172.21.15.96:0/2115579504 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (crc 0 0 0) 0x7fcc8000f2a0 con 0x7fcc8409b5d0 2024-03-21T06:20:25.222 INFO:teuthology.orchestra.run.smithi096.stdout:38654705670 2024-03-21T06:20:25.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.0 2024-03-21T06:20:25.223 DEBUG:teuthology.parallel:result is None 2024-03-21T06:20:25.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.224+0000 7fcc8ae64640 1 -- 172.21.15.96:0/2115579504 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcc70038300 msgr2=0x7fcc7003a810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.224+0000 7fcc8ae64640 1 --2- 172.21.15.96:0/2115579504 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcc70038300 0x7fcc7003a810 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.224+0000 7fcc8ae64640 1 -- 172.21.15.96:0/2115579504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc8409b5d0 msgr2=0x7fcc84128010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.224+0000 7fcc8ae64640 1 --2- 172.21.15.96:0/2115579504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc8409b5d0 0x7fcc84128010 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.224+0000 7fcc8ae64640 1 -- 172.21.15.96:0/2115579504 shutdown_connections 2024-03-21T06:20:25.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.224+0000 7fcc8ae64640 1 --2- 172.21.15.96:0/2115579504 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcc70038300 0x7fcc7003a810 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.224+0000 7fcc8ae64640 1 --2- 172.21.15.96:0/2115579504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc8409b5d0 0x7fcc84128010 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.224+0000 7fcc8ae64640 1 -- 172.21.15.96:0/2115579504 >> 172.21.15.96:0/2115579504 conn(0x7fcc8409fdb0 msgr2=0x7fcc84097900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:25.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.224+0000 7fcc8ae64640 1 -- 172.21.15.96:0/2115579504 shutdown_connections 2024-03-21T06:20:25.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.224+0000 7fcc8ae64640 1 -- 172.21.15.96:0/2115579504 wait complete. 2024-03-21T06:20:25.235 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.2 2024-03-21T06:20:25.235 DEBUG:teuthology.parallel:result is None 2024-03-21T06:20:25.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.245+0000 7fb259ffb640 1 -- 172.21.15.96:0/2296972601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb24c005300 con 0x7fb27409aa40 2024-03-21T06:20:25.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.245+0000 7fb25bfff640 1 -- 172.21.15.96:0/2296972601 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (crc 0 0 0) 0x7fb26400d610 con 0x7fb27409aa40 2024-03-21T06:20:25.246 INFO:teuthology.orchestra.run.smithi096.stdout:38654705670 2024-03-21T06:20:25.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.247+0000 7fb259ffb640 1 -- 172.21.15.96:0/2296972601 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb260038210 msgr2=0x7fb26003a720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.247+0000 7fb259ffb640 1 --2- 172.21.15.96:0/2296972601 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb260038210 0x7fb26003a720 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.247+0000 7fb259ffb640 1 -- 172.21.15.96:0/2296972601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27409aa40 msgr2=0x7fb274123f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.247+0000 7fb259ffb640 1 --2- 172.21.15.96:0/2296972601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27409aa40 0x7fb274123f60 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.248+0000 7fb259ffb640 1 -- 172.21.15.96:0/2296972601 shutdown_connections 2024-03-21T06:20:25.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.248+0000 7fb259ffb640 1 --2- 172.21.15.96:0/2296972601 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb260038210 0x7fb26003a720 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.248+0000 7fb259ffb640 1 --2- 172.21.15.96:0/2296972601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27409aa40 0x7fb274123f60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.248+0000 7fb259ffb640 1 -- 172.21.15.96:0/2296972601 >> 172.21.15.96:0/2296972601 conn(0x7fb27409d2f0 msgr2=0x7fb27411d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:25.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.248+0000 7fb259ffb640 1 -- 172.21.15.96:0/2296972601 shutdown_connections 2024-03-21T06:20:25.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.248+0000 7fb259ffb640 1 -- 172.21.15.96:0/2296972601 wait complete. 2024-03-21T06:20:25.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.252+0000 7fdb1acdc640 1 -- 172.21.15.96:0/2745762842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdb14073990 con 0x7fdb14098e20 2024-03-21T06:20:25.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.253+0000 7fdb117fa640 1 -- 172.21.15.96:0/2745762842 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (crc 0 0 0) 0x7fdb14073990 con 0x7fdb14098e20 2024-03-21T06:20:25.253 INFO:teuthology.orchestra.run.smithi096.stdout:38654705670 2024-03-21T06:20:25.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.255+0000 7fdb1acdc640 1 -- 172.21.15.96:0/2745762842 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fdaf80382b0 msgr2=0x7fdaf803a7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.255+0000 7fdb1acdc640 1 --2- 172.21.15.96:0/2745762842 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fdaf80382b0 0x7fdaf803a7c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.255+0000 7fdb1acdc640 1 -- 172.21.15.96:0/2745762842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb14098e20 msgr2=0x7fdb1412db10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.255+0000 7fdb1acdc640 1 --2- 172.21.15.96:0/2745762842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb14098e20 0x7fdb1412db10 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.255+0000 7fdb1acdc640 1 -- 172.21.15.96:0/2745762842 shutdown_connections 2024-03-21T06:20:25.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.255+0000 7fdb1acdc640 1 --2- 172.21.15.96:0/2745762842 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fdaf80382b0 0x7fdaf803a7c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.255+0000 7fdb1acdc640 1 --2- 172.21.15.96:0/2745762842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb14098e20 0x7fdb1412db10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.255+0000 7fdb1acdc640 1 -- 172.21.15.96:0/2745762842 >> 172.21.15.96:0/2745762842 conn(0x7fdb1409fdb0 msgr2=0x7fdb1406e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:25.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.255+0000 7fdb1acdc640 1 -- 172.21.15.96:0/2745762842 shutdown_connections 2024-03-21T06:20:25.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.255+0000 7fdb1acdc640 1 -- 172.21.15.96:0/2745762842 wait complete. 2024-03-21T06:20:25.259 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.3 2024-03-21T06:20:25.259 DEBUG:teuthology.parallel:result is None 2024-03-21T06:20:25.266 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.1 2024-03-21T06:20:25.266 DEBUG:teuthology.parallel:result is None 2024-03-21T06:20:25.266 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-21T06:20:25.266 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T06:20:25.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.408+0000 7f8baa86d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:25.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.413+0000 7f8baa86d640 1 Processor -- start 2024-03-21T06:20:25.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.413+0000 7f8baa86d640 1 -- start start 2024-03-21T06:20:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.413+0000 7f8baa86d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8ba409b360 0x7f8ba409cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.413+0000 7f8baa86d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba406e410 con 0x7f8ba41315d0 2024-03-21T06:20:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.413+0000 7f8baa86d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba409b770 con 0x7f8ba409b360 2024-03-21T06:20:25.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.413+0000 7f8ba3fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8ba409b360 0x7f8ba409cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.414+0000 7f8ba37fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8ba41315d0 0x7f8ba4134130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33178/0 (socket says 172.21.15.96:33178) 2024-03-21T06:20:25.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.414+0000 7f8ba37fe640 1 -- 172.21.15.96:0/1068381439 learned_addr learned my addr 172.21.15.96:0/1068381439 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:25.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.414+0000 7f8ba3fff640 1 -- 172.21.15.96:0/1068381439 >> v1:172.21.15.96:6789/0 conn(0x7f8ba41315d0 legacy=0x7f8ba4134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.414+0000 7f8ba3fff640 1 -- 172.21.15.96:0/1068381439 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba406dd00 con 0x7f8ba409b360 2024-03-21T06:20:25.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.414+0000 7f8ba3fff640 1 --2- 172.21.15.96:0/1068381439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba409b360 0x7f8ba409cda0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b197d29407e34e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.414+0000 7f8ba2ffd640 1 -- 172.21.15.96:0/1068381439 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8b98009ce0 con 0x7f8ba409b360 2024-03-21T06:20:25.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.414+0000 7f8ba2ffd640 1 -- 172.21.15.96:0/1068381439 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8b98009e40 con 0x7f8ba409b360 2024-03-21T06:20:25.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.414+0000 7f8ba2ffd640 1 -- 172.21.15.96:0/1068381439 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8b9800e360 con 0x7f8ba409b360 2024-03-21T06:20:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.415+0000 7f8baa86d640 1 -- 172.21.15.96:0/1068381439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba409b360 msgr2=0x7f8ba409cda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.415+0000 7f8baa86d640 1 --2- 172.21.15.96:0/1068381439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba409b360 0x7f8ba409cda0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.415+0000 7f8baa86d640 1 -- 172.21.15.96:0/1068381439 shutdown_connections 2024-03-21T06:20:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.415+0000 7f8baa86d640 1 --2- 172.21.15.96:0/1068381439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba409b360 0x7f8ba409cda0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.415+0000 7f8baa86d640 1 -- 172.21.15.96:0/1068381439 >> 172.21.15.96:0/1068381439 conn(0x7f8ba40a42c0 msgr2=0x7f8ba40a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.415+0000 7f8baa86d640 1 -- 172.21.15.96:0/1068381439 shutdown_connections 2024-03-21T06:20:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.415+0000 7f8baa86d640 1 -- 172.21.15.96:0/1068381439 wait complete. 2024-03-21T06:20:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.415+0000 7f8baa86d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.416+0000 7f8baa86d640 1 Processor -- start 2024-03-21T06:20:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.416+0000 7f8baa86d640 1 -- start start 2024-03-21T06:20:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.417+0000 7f8baa86d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba409b360 0x7f8ba411ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.417+0000 7f8baa86d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba4120740 con 0x7f8ba409b360 2024-03-21T06:20:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.417+0000 7f8ba3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba409b360 0x7f8ba411ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.417+0000 7f8ba3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba409b360 0x7f8ba411ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35270/0 (socket says 172.21.15.96:35270) 2024-03-21T06:20:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.417+0000 7f8ba3fff640 1 -- 172.21.15.96:0/2282272991 learned_addr learned my addr 172.21.15.96:0/2282272991 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.417+0000 7f8ba3fff640 1 -- 172.21.15.96:0/2282272991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba4130760 con 0x7f8ba409b360 2024-03-21T06:20:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.417+0000 7f8ba3fff640 1 --2- 172.21.15.96:0/2282272991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba409b360 0x7f8ba411ca10 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.417+0000 7f8ba0ff9640 1 -- 172.21.15.96:0/2282272991 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8b98014620 con 0x7f8ba409b360 2024-03-21T06:20:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.417+0000 7f8ba0ff9640 1 -- 172.21.15.96:0/2282272991 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f8b98014780 con 0x7f8ba409b360 2024-03-21T06:20:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.417+0000 7f8baa86d640 1 -- 172.21.15.96:0/2282272991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba411eb10 con 0x7f8ba409b360 2024-03-21T06:20:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.417+0000 7f8ba0ff9640 1 -- 172.21.15.96:0/2282272991 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f8b98014a20 con 0x7f8ba409b360 2024-03-21T06:20:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.417+0000 7f8baa86d640 1 -- 172.21.15.96:0/2282272991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba411fc40 con 0x7f8ba409b360 2024-03-21T06:20:25.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.418+0000 7f8ba0ff9640 1 -- 172.21.15.96:0/2282272991 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f8b98014b80 con 0x7f8ba409b360 2024-03-21T06:20:25.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.418+0000 7f8ba0ff9640 1 --2- 172.21.15.96:0/2282272991 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f8b90038260 0x7f8b9003a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.418+0000 7f8ba0ff9640 1 -- 172.21.15.96:0/2282272991 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f8b98019040 con 0x7f8ba409b360 2024-03-21T06:20:25.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.418+0000 7f8baa86d640 1 -- 172.21.15.96:0/2282272991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ba409ba50 con 0x7f8ba409b360 2024-03-21T06:20:25.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.418+0000 7f8ba37fe640 1 --2- 172.21.15.96:0/2282272991 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f8b90038260 0x7f8b9003a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.419+0000 7f8ba37fe640 1 --2- 172.21.15.96:0/2282272991 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f8b90038260 0x7f8b9003a770 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.423+0000 7f8ba0ff9640 1 -- 172.21.15.96:0/2282272991 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f8b98010310 con 0x7f8ba409b360 2024-03-21T06:20:25.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.592+0000 7f8baa86d640 1 -- 172.21.15.96:0/2282272991 --> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ba411eea0 con 0x7f8b90038260 2024-03-21T06:20:25.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.594+0000 7f8ba0ff9640 1 -- 172.21.15.96:0/2282272991 <== mgr.4099 v2:172.21.15.184:6800/2376803682 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+29954 (crc 0 0 0) 0x7f8ba411eea0 con 0x7f8b90038260 2024-03-21T06:20:25.596 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:20:25.597 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-03-21T06:20:25.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.596+0000 7f8baa86d640 1 -- 172.21.15.96:0/2282272991 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f8b90038260 msgr2=0x7f8b9003a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.596+0000 7f8baa86d640 1 --2- 172.21.15.96:0/2282272991 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f8b90038260 0x7f8b9003a770 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.596+0000 7f8baa86d640 1 -- 172.21.15.96:0/2282272991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba409b360 msgr2=0x7f8ba411ca10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.596+0000 7f8baa86d640 1 --2- 172.21.15.96:0/2282272991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba409b360 0x7f8ba411ca10 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.596+0000 7f8baa86d640 1 -- 172.21.15.96:0/2282272991 shutdown_connections 2024-03-21T06:20:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.596+0000 7f8baa86d640 1 --2- 172.21.15.96:0/2282272991 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f8b90038260 0x7f8b9003a770 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.596+0000 7f8baa86d640 1 --2- 172.21.15.96:0/2282272991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ba409b360 0x7f8ba411ca10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.596+0000 7f8baa86d640 1 -- 172.21.15.96:0/2282272991 >> 172.21.15.96:0/2282272991 conn(0x7f8ba40a42c0 msgr2=0x7f8ba4133f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.596+0000 7f8baa86d640 1 -- 172.21.15.96:0/2282272991 shutdown_connections 2024-03-21T06:20:25.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.596+0000 7f8baa86d640 1 -- 172.21.15.96:0/2282272991 wait complete. 2024-03-21T06:20:25.607 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-03-21T06:20:24.824743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":109776,"kb_used_data":1592,"kb_used_omap":0,"kb_used_meta":107776,"kb_avail":419320624,"statfs":{"total":429496729600,"available":429384318976,"internally_reserved":0,"allocated":1630208,"data_stored":1289732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":110362624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-21T06:20:23.723961+0000","last_change":"2024-03-21T06:20:23.724343+0000","last_active":"2024-03-21T06:20:23.723961+0000","last_peered":"2024-03-21T06:20:23.723961+0000","last_clean":"2024-03-21T06:20:22.674032+0000","last_became_active":"2024-03-21T06:20:23.721083+0000","last_became_peered":"2024-03-21T06:20:23.721083+0000","last_unstale":"2024-03-21T06:20:23.723961+0000","last_undegraded":"2024-03-21T06:20:23.723961+0000","last_fullsized":"2024-03-21T06:20:23.723961+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-21T06:20:23.719670+0000","last_change":"2024-03-21T06:20:23.720094+0000","last_active":"2024-03-21T06:20:23.719670+0000","last_peered":"2024-03-21T06:20:23.719670+0000","last_clean":"2024-03-21T06:20:22.674032+0000","last_became_active":"2024-03-21T06:20:23.717296+0000","last_became_peered":"2024-03-21T06:20:23.717296+0000","last_unstale":"2024-03-21T06:20:23.719670+0000","last_undegraded":"2024-03-21T06:20:23.719670+0000","last_fullsized":"2024-03-21T06:20:23.719670+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-03-21T06:20:24.694690+0000","last_change":"2024-03-21T06:20:23.724647+0000","last_active":"2024-03-21T06:20:24.694690+0000","last_peered":"2024-03-21T06:20:24.694690+0000","last_clean":"2024-03-21T06:20:24.694690+0000","last_became_active":"2024-03-21T06:20:23.721198+0000","last_became_peered":"2024-03-21T06:20:23.721198+0000","last_unstale":"2024-03-21T06:20:24.694690+0000","last_undegraded":"2024-03-21T06:20:24.694690+0000","last_fullsized":"2024-03-21T06:20:24.694690+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-21T06:20:23.716494+0000","last_change":"2024-03-21T06:20:23.716953+0000","last_active":"2024-03-21T06:20:23.716494+0000","last_peered":"2024-03-21T06:20:23.716494+0000","last_clean":"2024-03-21T06:20:22.674032+0000","last_became_active":"2024-03-21T06:20:23.714274+0000","last_became_peered":"2024-03-21T06:20:23.714274+0000","last_unstale":"2024-03-21T06:20:23.716494+0000","last_undegraded":"2024-03-21T06:20:23.716494+0000","last_fullsized":"2024-03-21T06:20:23.716494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-21T06:20:23.717077+0000","last_change":"2024-03-21T06:20:23.717551+0000","last_active":"2024-03-21T06:20:23.717077+0000","last_peered":"2024-03-21T06:20:23.717077+0000","last_clean":"2024-03-21T06:20:22.674032+0000","last_became_active":"2024-03-21T06:20:23.714766+0000","last_became_peered":"2024-03-21T06:20:23.714766+0000","last_unstale":"2024-03-21T06:20:23.717077+0000","last_undegraded":"2024-03-21T06:20:23.717077+0000","last_fullsized":"2024-03-21T06:20:23.717077+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-21T06:20:23.715548+0000","last_change":"2024-03-21T06:20:23.716172+0000","last_active":"2024-03-21T06:20:23.715548+0000","last_peered":"2024-03-21T06:20:23.715548+0000","last_clean":"2024-03-21T06:20:22.674032+0000","last_became_active":"2024-03-21T06:20:23.712781+0000","last_became_peered":"2024-03-21T06:20:23.712781+0000","last_unstale":"2024-03-21T06:20:23.715548+0000","last_undegraded":"2024-03-21T06:20:23.715548+0000","last_fullsized":"2024-03-21T06:20:23.715548+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-21T06:20:23.723623+0000","last_change":"2024-03-21T06:20:23.724311+0000","last_active":"2024-03-21T06:20:23.723623+0000","last_peered":"2024-03-21T06:20:23.723623+0000","last_clean":"2024-03-21T06:20:22.674032+0000","last_became_active":"2024-03-21T06:20:23.720894+0000","last_became_peered":"2024-03-21T06:20:23.720894+0000","last_unstale":"2024-03-21T06:20:23.723623+0000","last_undegraded":"2024-03-21T06:20:23.723623+0000","last_fullsized":"2024-03-21T06:20:23.723623+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-21T06:20:23.719710+0000","last_change":"2024-03-21T06:20:23.720175+0000","last_active":"2024-03-21T06:20:23.719710+0000","last_peered":"2024-03-21T06:20:23.719710+0000","last_clean":"2024-03-21T06:20:22.674032+0000","last_became_active":"2024-03-21T06:20:23.716768+0000","last_became_peered":"2024-03-21T06:20:23.716768+0000","last_unstale":"2024-03-21T06:20:23.719710+0000","last_undegraded":"2024-03-21T06:20:23.719710+0000","last_fullsized":"2024-03-21T06:20:23.719710+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":144,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-03-21T06:20:24.693188+0000","last_change":"2024-03-21T06:20:21.697506+0000","last_active":"2024-03-21T06:20:24.693188+0000","last_peered":"2024-03-21T06:20:24.693188+0000","last_clean":"2024-03-21T06:20:24.693188+0000","last_became_active":"2024-03-21T06:20:21.690580+0000","last_became_peered":"2024-03-21T06:20:21.690580+0000","last_unstale":"2024-03-21T06:20:24.693188+0000","last_undegraded":"2024-03-21T06:20:24.693188+0000","last_fullsized":"2024-03-21T06:20:24.693188+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:20.658895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:20.658895+0000","last_clean_scrub_stamp":"2024-03-21T06:20:20.658895+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27828,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829772,"statfs":{"total":107374182400,"available":107345686528,"internally_reserved":0,"allocated":704512,"data_stored":617617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27060,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830540,"statfs":{"total":107374182400,"available":107346472960,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27060,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830540,"statfs":{"total":107374182400,"available":107346472960,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27828,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829772,"statfs":{"total":107374182400,"available":107345686528,"internally_reserved":0,"allocated":704512,"data_stored":617617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-21T06:20:25.608 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T06:20:25.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.711+0000 7f0b3251c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:25.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.715+0000 7f0b3251c640 1 Processor -- start 2024-03-21T06:20:25.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.715+0000 7f0b3251c640 1 -- start start 2024-03-21T06:20:25.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.716+0000 7f0b3251c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0b2c09cda0 0x7f0b2c139450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.716+0000 7f0b3251c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b2c06e410 con 0x7f0b2c09b360 2024-03-21T06:20:25.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.716+0000 7f0b3251c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b2c09b770 con 0x7f0b2c09cda0 2024-03-21T06:20:25.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.716+0000 7f0b2b7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0b2c09cda0 0x7f0b2c139450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.716+0000 7f0b2bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f0b2c09b360 0x7f0b2c1315d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44118/0 (socket says 172.21.15.96:44118) 2024-03-21T06:20:25.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.716+0000 7f0b2bfff640 1 -- 172.21.15.96:0/517556583 learned_addr learned my addr 172.21.15.96:0/517556583 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:25.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.717+0000 7f0b2b7fe640 1 -- 172.21.15.96:0/517556583 >> v1:172.21.15.96:6789/0 conn(0x7f0b2c09b360 legacy=0x7f0b2c1315d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.717+0000 7f0b2b7fe640 1 -- 172.21.15.96:0/517556583 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c09d1b0 con 0x7f0b2c09cda0 2024-03-21T06:20:25.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.717+0000 7f0b2b7fe640 1 --2- 172.21.15.96:0/517556583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c09cda0 0x7f0b2c139450 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83b7ff913b32953 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.717+0000 7f0b2affd640 1 -- 172.21.15.96:0/517556583 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0b20009ce0 con 0x7f0b2c09cda0 2024-03-21T06:20:25.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.717+0000 7f0b2affd640 1 -- 172.21.15.96:0/517556583 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0b20009e40 con 0x7f0b2c09cda0 2024-03-21T06:20:25.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.717+0000 7f0b2affd640 1 -- 172.21.15.96:0/517556583 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0b2000e360 con 0x7f0b2c09cda0 2024-03-21T06:20:25.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.717+0000 7f0b3251c640 1 -- 172.21.15.96:0/517556583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c09cda0 msgr2=0x7f0b2c139450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.717+0000 7f0b3251c640 1 --2- 172.21.15.96:0/517556583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c09cda0 0x7f0b2c139450 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.718+0000 7f0b3251c640 1 -- 172.21.15.96:0/517556583 shutdown_connections 2024-03-21T06:20:25.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.718+0000 7f0b3251c640 1 --2- 172.21.15.96:0/517556583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c09cda0 0x7f0b2c139450 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.718+0000 7f0b3251c640 1 -- 172.21.15.96:0/517556583 >> 172.21.15.96:0/517556583 conn(0x7f0b2c0a42c0 msgr2=0x7f0b2c0a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:25.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.718+0000 7f0b3251c640 1 -- 172.21.15.96:0/517556583 shutdown_connections 2024-03-21T06:20:25.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.718+0000 7f0b3251c640 1 -- 172.21.15.96:0/517556583 wait complete. 2024-03-21T06:20:25.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.718+0000 7f0b3251c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:25.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.719+0000 7f0b3251c640 1 Processor -- start 2024-03-21T06:20:25.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.719+0000 7f0b3251c640 1 -- start start 2024-03-21T06:20:25.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.719+0000 7f0b3251c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c09b360 0x7f0b2c1c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.719+0000 7f0b3251c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c11e420 con 0x7f0b2c09b360 2024-03-21T06:20:25.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.720+0000 7f0b2bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c09b360 0x7f0b2c1c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.720+0000 7f0b2bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c09b360 0x7f0b2c1c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48770/0 (socket says 172.21.15.96:48770) 2024-03-21T06:20:25.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.720+0000 7f0b2bfff640 1 -- 172.21.15.96:0/2353260335 learned_addr learned my addr 172.21.15.96:0/2353260335 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:25.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.720+0000 7f0b2bfff640 1 -- 172.21.15.96:0/2353260335 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c073030 con 0x7f0b2c09b360 2024-03-21T06:20:25.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.720+0000 7f0b2bfff640 1 --2- 172.21.15.96:0/2353260335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c09b360 0x7f0b2c1c8d20 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.720+0000 7f0b28ff9640 1 -- 172.21.15.96:0/2353260335 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0b1c0059d0 con 0x7f0b2c09b360 2024-03-21T06:20:25.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.720+0000 7f0b28ff9640 1 -- 172.21.15.96:0/2353260335 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0b1c005b30 con 0x7f0b2c09b360 2024-03-21T06:20:25.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.720+0000 7f0b3251c640 1 -- 172.21.15.96:0/2353260335 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b2c11cd30 con 0x7f0b2c09b360 2024-03-21T06:20:25.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.720+0000 7f0b28ff9640 1 -- 172.21.15.96:0/2353260335 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0b1c005dd0 con 0x7f0b2c09b360 2024-03-21T06:20:25.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.720+0000 7f0b3251c640 1 -- 172.21.15.96:0/2353260335 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b2c11d510 con 0x7f0b2c09b360 2024-03-21T06:20:25.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.721+0000 7f0b28ff9640 1 -- 172.21.15.96:0/2353260335 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f0b1c0070b0 con 0x7f0b2c09b360 2024-03-21T06:20:25.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.721+0000 7f0b3251c640 1 -- 172.21.15.96:0/2353260335 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b2c11cf20 con 0x7f0b2c09b360 2024-03-21T06:20:25.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.721+0000 7f0b28ff9640 1 --2- 172.21.15.96:0/2353260335 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f0b18038260 0x7f0b1803a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:25.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.721+0000 7f0b28ff9640 1 -- 172.21.15.96:0/2353260335 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f0b1c047630 con 0x7f0b2c09b360 2024-03-21T06:20:25.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.721+0000 7f0b2b7fe640 1 --2- 172.21.15.96:0/2353260335 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f0b18038260 0x7f0b1803a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:25.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.722+0000 7f0b2b7fe640 1 --2- 172.21.15.96:0/2353260335 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f0b18038260 0x7f0b1803a770 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:25.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.726+0000 7f0b28ff9640 1 -- 172.21.15.96:0/2353260335 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f0b2c11cf20 con 0x7f0b2c09b360 2024-03-21T06:20:25.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.896+0000 7f0b3251c640 1 -- 172.21.15.96:0/2353260335 --> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b2c09ba80 con 0x7f0b18038260 2024-03-21T06:20:25.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.897+0000 7f0b28ff9640 1 -- 172.21.15.96:0/2353260335 <== mgr.4099 v2:172.21.15.184:6800/2376803682 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+29954 (crc 0 0 0) 0x7f0b200124e0 con 0x7f0b18038260 2024-03-21T06:20:25.898 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:20:25.898 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-03-21T06:20:25.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.899+0000 7f0b3251c640 1 -- 172.21.15.96:0/2353260335 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f0b18038260 msgr2=0x7f0b1803a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.899+0000 7f0b3251c640 1 --2- 172.21.15.96:0/2353260335 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f0b18038260 0x7f0b1803a770 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.899+0000 7f0b3251c640 1 -- 172.21.15.96:0/2353260335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c09b360 msgr2=0x7f0b2c1c8d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:25.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.899+0000 7f0b3251c640 1 --2- 172.21.15.96:0/2353260335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c09b360 0x7f0b2c1c8d20 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.899+0000 7f0b3251c640 1 -- 172.21.15.96:0/2353260335 shutdown_connections 2024-03-21T06:20:25.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.899+0000 7f0b3251c640 1 --2- 172.21.15.96:0/2353260335 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f0b18038260 0x7f0b1803a770 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.899+0000 7f0b3251c640 1 --2- 172.21.15.96:0/2353260335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b2c09b360 0x7f0b2c1c8d20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:25.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.899+0000 7f0b3251c640 1 -- 172.21.15.96:0/2353260335 >> 172.21.15.96:0/2353260335 conn(0x7f0b2c0a42c0 msgr2=0x7f0b2c0a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:25.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.899+0000 7f0b3251c640 1 -- 172.21.15.96:0/2353260335 shutdown_connections 2024-03-21T06:20:25.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:25.899+0000 7f0b3251c640 1 -- 172.21.15.96:0/2353260335 wait complete. 2024-03-21T06:20:25.910 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-03-21T06:20:24.824743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":109776,"kb_used_data":1592,"kb_used_omap":0,"kb_used_meta":107776,"kb_avail":419320624,"statfs":{"total":429496729600,"available":429384318976,"internally_reserved":0,"allocated":1630208,"data_stored":1289732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":110362624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-21T06:20:23.723961+0000","last_change":"2024-03-21T06:20:23.724343+0000","last_active":"2024-03-21T06:20:23.723961+0000","last_peered":"2024-03-21T06:20:23.723961+0000","last_clean":"2024-03-21T06:20:22.674032+0000","last_became_active":"2024-03-21T06:20:23.721083+0000","last_became_peered":"2024-03-21T06:20:23.721083+0000","last_unstale":"2024-03-21T06:20:23.723961+0000","last_undegraded":"2024-03-21T06:20:23.723961+0000","last_fullsized":"2024-03-21T06:20:23.723961+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-21T06:20:23.719670+0000","last_change":"2024-03-21T06:20:23.720094+0000","last_active":"2024-03-21T06:20:23.719670+0000","last_peered":"2024-03-21T06:20:23.719670+0000","last_clean":"2024-03-21T06:20:22.674032+0000","last_became_active":"2024-03-21T06:20:23.717296+0000","last_became_peered":"2024-03-21T06:20:23.717296+0000","last_unstale":"2024-03-21T06:20:23.719670+0000","last_undegraded":"2024-03-21T06:20:23.719670+0000","last_fullsized":"2024-03-21T06:20:23.719670+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-03-21T06:20:24.694690+0000","last_change":"2024-03-21T06:20:23.724647+0000","last_active":"2024-03-21T06:20:24.694690+0000","last_peered":"2024-03-21T06:20:24.694690+0000","last_clean":"2024-03-21T06:20:24.694690+0000","last_became_active":"2024-03-21T06:20:23.721198+0000","last_became_peered":"2024-03-21T06:20:23.721198+0000","last_unstale":"2024-03-21T06:20:24.694690+0000","last_undegraded":"2024-03-21T06:20:24.694690+0000","last_fullsized":"2024-03-21T06:20:24.694690+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-21T06:20:23.716494+0000","last_change":"2024-03-21T06:20:23.716953+0000","last_active":"2024-03-21T06:20:23.716494+0000","last_peered":"2024-03-21T06:20:23.716494+0000","last_clean":"2024-03-21T06:20:22.674032+0000","last_became_active":"2024-03-21T06:20:23.714274+0000","last_became_peered":"2024-03-21T06:20:23.714274+0000","last_unstale":"2024-03-21T06:20:23.716494+0000","last_undegraded":"2024-03-21T06:20:23.716494+0000","last_fullsized":"2024-03-21T06:20:23.716494+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-21T06:20:23.717077+0000","last_change":"2024-03-21T06:20:23.717551+0000","last_active":"2024-03-21T06:20:23.717077+0000","last_peered":"2024-03-21T06:20:23.717077+0000","last_clean":"2024-03-21T06:20:22.674032+0000","last_became_active":"2024-03-21T06:20:23.714766+0000","last_became_peered":"2024-03-21T06:20:23.714766+0000","last_unstale":"2024-03-21T06:20:23.717077+0000","last_undegraded":"2024-03-21T06:20:23.717077+0000","last_fullsized":"2024-03-21T06:20:23.717077+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-21T06:20:23.715548+0000","last_change":"2024-03-21T06:20:23.716172+0000","last_active":"2024-03-21T06:20:23.715548+0000","last_peered":"2024-03-21T06:20:23.715548+0000","last_clean":"2024-03-21T06:20:22.674032+0000","last_became_active":"2024-03-21T06:20:23.712781+0000","last_became_peered":"2024-03-21T06:20:23.712781+0000","last_unstale":"2024-03-21T06:20:23.715548+0000","last_undegraded":"2024-03-21T06:20:23.715548+0000","last_fullsized":"2024-03-21T06:20:23.715548+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-21T06:20:23.723623+0000","last_change":"2024-03-21T06:20:23.724311+0000","last_active":"2024-03-21T06:20:23.723623+0000","last_peered":"2024-03-21T06:20:23.723623+0000","last_clean":"2024-03-21T06:20:22.674032+0000","last_became_active":"2024-03-21T06:20:23.720894+0000","last_became_peered":"2024-03-21T06:20:23.720894+0000","last_unstale":"2024-03-21T06:20:23.723623+0000","last_undegraded":"2024-03-21T06:20:23.723623+0000","last_fullsized":"2024-03-21T06:20:23.723623+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-03-21T06:20:23.719710+0000","last_change":"2024-03-21T06:20:23.720175+0000","last_active":"2024-03-21T06:20:23.719710+0000","last_peered":"2024-03-21T06:20:23.719710+0000","last_clean":"2024-03-21T06:20:22.674032+0000","last_became_active":"2024-03-21T06:20:23.716768+0000","last_became_peered":"2024-03-21T06:20:23.716768+0000","last_unstale":"2024-03-21T06:20:23.719710+0000","last_undegraded":"2024-03-21T06:20:23.719710+0000","last_fullsized":"2024-03-21T06:20:23.719710+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":144,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-03-21T06:20:24.693188+0000","last_change":"2024-03-21T06:20:21.697506+0000","last_active":"2024-03-21T06:20:24.693188+0000","last_peered":"2024-03-21T06:20:24.693188+0000","last_clean":"2024-03-21T06:20:24.693188+0000","last_became_active":"2024-03-21T06:20:21.690580+0000","last_became_peered":"2024-03-21T06:20:21.690580+0000","last_unstale":"2024-03-21T06:20:24.693188+0000","last_undegraded":"2024-03-21T06:20:24.693188+0000","last_fullsized":"2024-03-21T06:20:24.693188+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:20.658895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:20.658895+0000","last_clean_scrub_stamp":"2024-03-21T06:20:20.658895+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27828,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829772,"statfs":{"total":107374182400,"available":107345686528,"internally_reserved":0,"allocated":704512,"data_stored":617617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27060,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830540,"statfs":{"total":107374182400,"available":107346472960,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27060,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104830540,"statfs":{"total":107374182400,"available":107346472960,"internally_reserved":0,"allocated":110592,"data_stored":27249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27828,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":104829772,"statfs":{"total":107374182400,"available":107345686528,"internally_reserved":0,"allocated":704512,"data_stored":617617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-21T06:20:25.911 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-21T06:20:25.911 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-21T06:20:25.911 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-21T06:20:25.911 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-21T06:20:26.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.012+0000 7fc19dd5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:26.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.015+0000 7fc19dd5c640 1 Processor -- start 2024-03-21T06:20:26.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.015+0000 7fc19dd5c640 1 -- start start 2024-03-21T06:20:26.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.016+0000 7fc19dd5c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc19809c810 0x7fc19809cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:26.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.016+0000 7fc19dd5c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19809aa40 con 0x7fc1981341b0 2024-03-21T06:20:26.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.016+0000 7fc19dd5c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19809c410 con 0x7fc19809c810 2024-03-21T06:20:26.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.016+0000 7fc1977fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc19809c810 0x7fc19809cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:26.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.016+0000 7fc196ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fc1981341b0 0x7fc1981345c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44128/0 (socket says 172.21.15.96:44128) 2024-03-21T06:20:26.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.016+0000 7fc196ffd640 1 -- 172.21.15.96:0/602078163 learned_addr learned my addr 172.21.15.96:0/602078163 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:20:26.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.016+0000 7fc1977fe640 1 -- 172.21.15.96:0/602078163 >> v1:172.21.15.96:6789/0 conn(0x7fc1981341b0 legacy=0x7fc1981345c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.016+0000 7fc1977fe640 1 -- 172.21.15.96:0/602078163 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19809b030 con 0x7fc19809c810 2024-03-21T06:20:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.016+0000 7fc1967fc640 1 -- 172.21.15.96:0/602078163 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175723898 0 0) 0x7fc19809aa40 con 0x7fc1981341b0 2024-03-21T06:20:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.016+0000 7fc1977fe640 1 --2- 172.21.15.96:0/602078163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc19809c810 0x7fc19809cc20 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3700084aeb0fb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.016+0000 7fc1967fc640 1 -- 172.21.15.96:0/602078163 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc188009ce0 con 0x7fc19809c810 2024-03-21T06:20:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.016+0000 7fc1967fc640 1 -- 172.21.15.96:0/602078163 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc188009e40 con 0x7fc19809c810 2024-03-21T06:20:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.016+0000 7fc1967fc640 1 -- 172.21.15.96:0/602078163 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc18800e360 con 0x7fc19809c810 2024-03-21T06:20:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.017+0000 7fc19dd5c640 1 -- 172.21.15.96:0/602078163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc19809c810 msgr2=0x7fc19809cc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.017+0000 7fc19dd5c640 1 --2- 172.21.15.96:0/602078163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc19809c810 0x7fc19809cc20 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.017+0000 7fc19dd5c640 1 -- 172.21.15.96:0/602078163 shutdown_connections 2024-03-21T06:20:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.017+0000 7fc19dd5c640 1 --2- 172.21.15.96:0/602078163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc19809c810 0x7fc19809cc20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.017+0000 7fc19dd5c640 1 -- 172.21.15.96:0/602078163 >> 172.21.15.96:0/602078163 conn(0x7fc1980a41b0 msgr2=0x7fc1980a45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:26.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.017+0000 7fc19dd5c640 1 -- 172.21.15.96:0/602078163 shutdown_connections 2024-03-21T06:20:26.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.017+0000 7fc19dd5c640 1 -- 172.21.15.96:0/602078163 wait complete. 2024-03-21T06:20:26.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.017+0000 7fc19dd5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:20:26.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.018+0000 7fc19dd5c640 1 Processor -- start 2024-03-21T06:20:26.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.019+0000 7fc19dd5c640 1 -- start start 2024-03-21T06:20:26.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.019+0000 7fc19dd5c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc19809c810 0x7fc19811dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:26.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.019+0000 7fc19dd5c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19811d790 con 0x7fc19809c810 2024-03-21T06:20:26.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.019+0000 7fc1977fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc19809c810 0x7fc19811dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:26.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.019+0000 7fc1977fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc19809c810 0x7fc19811dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48786/0 (socket says 172.21.15.96:48786) 2024-03-21T06:20:26.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.019+0000 7fc1977fe640 1 -- 172.21.15.96:0/882831164 learned_addr learned my addr 172.21.15.96:0/882831164 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:20:26.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.020+0000 7fc1977fe640 1 -- 172.21.15.96:0/882831164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19811ee10 con 0x7fc19809c810 2024-03-21T06:20:26.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.020+0000 7fc1977fe640 1 --2- 172.21.15.96:0/882831164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc19809c810 0x7fc19811dd80 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:26.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.020+0000 7fc17ffff640 1 -- 172.21.15.96:0/882831164 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc188014620 con 0x7fc19809c810 2024-03-21T06:20:26.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.020+0000 7fc17ffff640 1 -- 172.21.15.96:0/882831164 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc188014780 con 0x7fc19809c810 2024-03-21T06:20:26.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.020+0000 7fc19dd5c640 1 -- 172.21.15.96:0/882831164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19811f650 con 0x7fc19809c810 2024-03-21T06:20:26.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.020+0000 7fc17ffff640 1 -- 172.21.15.96:0/882831164 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc188014a20 con 0x7fc19809c810 2024-03-21T06:20:26.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.020+0000 7fc19dd5c640 1 -- 172.21.15.96:0/882831164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc198120680 con 0x7fc19809c810 2024-03-21T06:20:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.021+0000 7fc17ffff640 1 -- 172.21.15.96:0/882831164 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fc188014bd0 con 0x7fc19809c810 2024-03-21T06:20:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.021+0000 7fc19dd5c640 1 -- 172.21.15.96:0/882831164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc198073c60 con 0x7fc19809c810 2024-03-21T06:20:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.021+0000 7fc17ffff640 1 --2- 172.21.15.96:0/882831164 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc184038260 0x7fc18403a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:20:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.021+0000 7fc17ffff640 1 -- 172.21.15.96:0/882831164 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2672+0+0 (crc 0 0 0) 0x7fc188019040 con 0x7fc19809c810 2024-03-21T06:20:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.021+0000 7fc196ffd640 1 --2- 172.21.15.96:0/882831164 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc184038260 0x7fc18403a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:20:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.022+0000 7fc196ffd640 1 --2- 172.21.15.96:0/882831164 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc184038260 0x7fc18403a770 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:20:26.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.026+0000 7fc17ffff640 1 -- 172.21.15.96:0/882831164 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fc188018190 con 0x7fc19809c810 2024-03-21T06:20:26.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.235+0000 7fc19dd5c640 1 -- 172.21.15.96:0/882831164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc198074b50 con 0x7fc19809c810 2024-03-21T06:20:26.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.236+0000 7fc17ffff640 1 -- 172.21.15.96:0/882831164 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (crc 0 0 0) 0x7fc1880067d0 con 0x7fc19809c810 2024-03-21T06:20:26.237 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:20:26.237 INFO:teuthology.orchestra.run.smithi096.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-21T06:20:26.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.238+0000 7fc19dd5c640 1 -- 172.21.15.96:0/882831164 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc184038260 msgr2=0x7fc18403a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.238+0000 7fc19dd5c640 1 --2- 172.21.15.96:0/882831164 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc184038260 0x7fc18403a770 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.238+0000 7fc19dd5c640 1 -- 172.21.15.96:0/882831164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc19809c810 msgr2=0x7fc19811dd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:20:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.238+0000 7fc19dd5c640 1 --2- 172.21.15.96:0/882831164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc19809c810 0x7fc19811dd80 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.238+0000 7fc19dd5c640 1 -- 172.21.15.96:0/882831164 shutdown_connections 2024-03-21T06:20:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.238+0000 7fc19dd5c640 1 --2- 172.21.15.96:0/882831164 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc184038260 0x7fc18403a770 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.238+0000 7fc19dd5c640 1 --2- 172.21.15.96:0/882831164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc19809c810 0x7fc19811dd80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:20:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.238+0000 7fc19dd5c640 1 -- 172.21.15.96:0/882831164 >> 172.21.15.96:0/882831164 conn(0x7fc1980a41b0 msgr2=0x7fc1981ba220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:20:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.238+0000 7fc19dd5c640 1 -- 172.21.15.96:0/882831164 shutdown_connections 2024-03-21T06:20:26.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:20:26.238+0000 7fc19dd5c640 1 -- 172.21.15.96:0/882831164 wait complete. 2024-03-21T06:20:26.249 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-21T06:20:26.249 INFO:teuthology.run_tasks:Running task workunit... 2024-03-21T06:20:26.266 INFO:tasks.workunit:Pulling workunits from ref 63e11116edc7c1585450815ed7030c70ea0ebef9 2024-03-21T06:20:26.266 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-21T06:20:26.267 DEBUG:teuthology.orchestra.run.smithi096:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-21T06:20:26.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T06:20:26.282 INFO:teuthology.orchestra.run.smithi096.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-21T06:20:26.282 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-21T06:20:26.336 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-21T06:20:26.336 DEBUG:teuthology.orchestra.run.smithi096:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-21T06:20:26.391 INFO:tasks.workunit:timeout=3h 2024-03-21T06:20:26.391 INFO:tasks.workunit:cleanup=True 2024-03-21T06:20:26.391 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 63e11116edc7c1585450815ed7030c70ea0ebef9 2024-03-21T06:20:26.446 INFO:tasks.workunit.client.0.smithi096.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-21T06:21:57.034 INFO:tasks.workunit.client.0.smithi096.stderr:Note: switching to '63e11116edc7c1585450815ed7030c70ea0ebef9'. 2024-03-21T06:21:57.034 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-03-21T06:21:57.034 INFO:tasks.workunit.client.0.smithi096.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-21T06:21:57.035 INFO:tasks.workunit.client.0.smithi096.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-21T06:21:57.035 INFO:tasks.workunit.client.0.smithi096.stderr:state without impacting any branches by switching back to a branch. 2024-03-21T06:21:57.035 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-03-21T06:21:57.035 INFO:tasks.workunit.client.0.smithi096.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-21T06:21:57.035 INFO:tasks.workunit.client.0.smithi096.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-21T06:21:57.035 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-03-21T06:21:57.035 INFO:tasks.workunit.client.0.smithi096.stderr: git switch -c 2024-03-21T06:21:57.035 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-03-21T06:21:57.035 INFO:tasks.workunit.client.0.smithi096.stderr:Or undo this operation with: 2024-03-21T06:21:57.035 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-03-21T06:21:57.035 INFO:tasks.workunit.client.0.smithi096.stderr: git switch - 2024-03-21T06:21:57.035 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-03-21T06:21:57.035 INFO:tasks.workunit.client.0.smithi096.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-21T06:21:57.035 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-03-21T06:21:57.036 INFO:tasks.workunit.client.0.smithi096.stderr:HEAD is now at 63e11116edc empty 2024-03-21T06:21:57.038 DEBUG:teuthology.orchestra.run.smithi096:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-21T06:21:57.105 INFO:tasks.workunit.client.0.smithi096.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-21T06:21:57.107 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T06:21:57.107 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-21T06:21:57.868 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-21T06:21:57.942 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-21T06:21:57.983 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T06:21:57.984 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T06:21:57.984 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-21T06:21:58.024 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T06:21:58.027 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-21T06:21:58.027 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-21T06:21:58.082 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-03-21T06:21:58.084 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-03-21T06:21:58.084 DEBUG:teuthology.orchestra.run.smithi096:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=63e11116edc7c1585450815ed7030c70ea0ebef9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson 2024-03-21T06:21:58.141 INFO:tasks.workunit.client.0.smithi096.stderr:+ parallel=1 2024-03-21T06:21:58.141 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' --crimson = --serial ']' 2024-03-21T06:21:58.141 INFO:tasks.workunit.client.0.smithi096.stderr:+ crimson=0 2024-03-21T06:21:58.142 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' --crimson = --crimson ']' 2024-03-21T06:21:58.142 INFO:tasks.workunit.client.0.smithi096.stderr:+ parallel=0 2024-03-21T06:21:58.142 INFO:tasks.workunit.client.0.smithi096.stderr:+ crimson=1 2024-03-21T06:21:58.142 INFO:tasks.workunit.client.0.smithi096.stderr:+ color= 2024-03-21T06:21:58.142 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' -t 1 ']' 2024-03-21T06:21:58.142 INFO:tasks.workunit.client.0.smithi096.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-03-21T06:21:58.142 INFO:tasks.workunit.client.0.smithi096.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-21T06:21:58.142 INFO:tasks.workunit.client.0.smithi096.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-21T06:21:58.143 INFO:tasks.workunit.client.0.smithi096.stderr:+ declare -A pids 2024-03-21T06:21:58.143 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:21:58.143 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:21:58.143 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_aio 2024-03-21T06:21:58.157 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:21:58.157 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 42 tests from 2 test suites. 2024-03-21T06:21:58.157 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:21:58.157 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 26 tests from LibRadosAio 2024-03-21T06:21:58.157 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.TooBig 2024-03-21T06:22:00.695 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.TooBig (2537 ms) 2024-03-21T06:22:00.695 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.SimpleWrite 2024-03-21T06:22:03.712 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.SimpleWrite (3017 ms) 2024-03-21T06:22:03.712 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.WaitForSafe 2024-03-21T06:22:06.727 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.WaitForSafe (3016 ms) 2024-03-21T06:22:06.728 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RoundTrip 2024-03-21T06:22:09.747 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RoundTrip (3020 ms) 2024-03-21T06:22:09.747 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RoundTrip2 2024-03-21T06:22:12.765 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RoundTrip2 (3018 ms) 2024-03-21T06:22:12.765 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RoundTrip3 2024-03-21T06:22:15.780 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RoundTrip3 (3014 ms) 2024-03-21T06:22:15.780 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RoundTripAppend 2024-03-21T06:22:18.789 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RoundTripAppend (3010 ms) 2024-03-21T06:22:18.790 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RemoveTest 2024-03-21T06:22:21.799 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RemoveTest (3010 ms) 2024-03-21T06:22:21.799 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.XattrsRoundTrip 2024-03-21T06:22:24.810 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.XattrsRoundTrip (3011 ms) 2024-03-21T06:22:24.811 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RmXattr 2024-03-21T06:22:27.827 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RmXattr (3017 ms) 2024-03-21T06:22:27.828 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.XattrIter 2024-03-21T06:22:30.847 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.XattrIter (3020 ms) 2024-03-21T06:22:30.848 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.IsComplete 2024-03-21T06:22:33.857 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.IsComplete (3009 ms) 2024-03-21T06:22:33.857 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.IsSafe 2024-03-21T06:22:36.872 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.IsSafe (3016 ms) 2024-03-21T06:22:36.873 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.ReturnValue 2024-03-21T06:22:39.882 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.ReturnValue (3010 ms) 2024-03-21T06:22:39.883 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.Flush 2024-03-21T06:22:42.902 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.Flush (3016 ms) 2024-03-21T06:22:42.902 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.FlushAsync 2024-03-21T06:22:45.911 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.FlushAsync (3012 ms) 2024-03-21T06:22:45.911 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RoundTripWriteFull 2024-03-21T06:22:48.928 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RoundTripWriteFull (3017 ms) 2024-03-21T06:22:48.928 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RoundTripWriteSame 2024-03-21T06:22:51.942 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RoundTripWriteSame (3015 ms) 2024-03-21T06:22:51.942 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.SimpleStat 2024-03-21T06:22:54.292 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.SimpleStat (2350 ms) 2024-03-21T06:22:54.293 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.OperateMtime 2024-03-21T06:22:57.290 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.OperateMtime (2997 ms) 2024-03-21T06:22:57.290 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.Operate2Mtime 2024-03-21T06:23:00.322 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.Operate2Mtime (3032 ms) 2024-03-21T06:23:00.322 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.SimpleStatNS 2024-03-21T06:23:03.343 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.SimpleStatNS (3021 ms) 2024-03-21T06:23:03.343 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.StatRemove 2024-03-21T06:23:06.350 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.StatRemove (3008 ms) 2024-03-21T06:23:06.350 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.ExecuteClass 2024-03-21T06:23:09.294 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.ExecuteClass (2944 ms) 2024-03-21T06:23:09.294 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.MultiWrite 2024-03-21T06:23:12.301 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.MultiWrite (3007 ms) 2024-03-21T06:23:12.301 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.AioUnlock 2024-03-21T06:23:15.317 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.AioUnlock (3014 ms) 2024-03-21T06:23:15.317 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 26 tests from LibRadosAio (77158 ms total) 2024-03-21T06:23:15.318 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:23:15.318 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 16 tests from LibRadosAioEC 2024-03-21T06:23:15.318 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.SimpleWrite 2024-03-21T06:23:15.318 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio.cc:1100: Skipped 2024-03-21T06:23:15.318 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:23:15.318 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWrite (0 ms) 2024-03-21T06:23:15.318 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.WaitForComplete 2024-03-21T06:23:15.318 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio.cc:1132: Skipped 2024-03-21T06:23:15.318 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:23:15.319 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.WaitForComplete (0 ms) 2024-03-21T06:23:15.319 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.RoundTrip 2024-03-21T06:23:15.319 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio.cc:1149: Skipped 2024-03-21T06:23:15.319 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:23:15.319 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip (0 ms) 2024-03-21T06:23:15.319 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.RoundTrip2 2024-03-21T06:23:15.319 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio.cc:1182: Skipped 2024-03-21T06:23:15.319 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:23:15.319 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip2 (0 ms) 2024-03-21T06:23:15.319 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.RoundTripAppend 2024-03-21T06:23:15.319 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio.cc:1215: Skipped 2024-03-21T06:23:15.320 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:23:15.320 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppend (0 ms) 2024-03-21T06:23:15.320 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.IsComplete 2024-03-21T06:23:15.320 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio.cc:1286: Skipped 2024-03-21T06:23:15.320 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:23:15.320 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.IsComplete (0 ms) 2024-03-21T06:23:15.320 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.IsSafe 2024-03-21T06:23:15.320 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio.cc:1326: Skipped 2024-03-21T06:23:15.320 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:23:15.320 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.IsSafe (0 ms) 2024-03-21T06:23:15.320 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.ReturnValue 2024-03-21T06:23:15.321 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio.cc:1366: Skipped 2024-03-21T06:23:15.321 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:23:15.321 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValue (0 ms) 2024-03-21T06:23:15.321 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.Flush 2024-03-21T06:23:15.321 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio.cc:1385: Skipped 2024-03-21T06:23:15.321 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:23:15.321 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.Flush (0 ms) 2024-03-21T06:23:15.321 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.FlushAsync 2024-03-21T06:23:15.321 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio.cc:1415: Skipped 2024-03-21T06:23:15.321 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:23:15.322 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsync (0 ms) 2024-03-21T06:23:15.322 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-03-21T06:23:15.322 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio.cc:1454: Skipped 2024-03-21T06:23:15.322 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:23:15.322 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFull (0 ms) 2024-03-21T06:23:15.322 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.SimpleStat 2024-03-21T06:23:15.322 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio.cc:1498: Skipped 2024-03-21T06:23:15.322 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:23:15.322 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStat (0 ms) 2024-03-21T06:23:15.323 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.SimpleStatNS 2024-03-21T06:23:15.323 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio.cc:1532: Skipped 2024-03-21T06:23:15.323 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:23:15.323 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatNS (0 ms) 2024-03-21T06:23:15.323 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.StatRemove 2024-03-21T06:23:15.323 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio.cc:1593: Skipped 2024-03-21T06:23:15.323 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:23:15.323 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.StatRemove (0 ms) 2024-03-21T06:23:15.323 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.ExecuteClass 2024-03-21T06:23:15.323 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio.cc:1649: Skipped 2024-03-21T06:23:15.323 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:23:15.323 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClass (0 ms) 2024-03-21T06:23:15.323 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.MultiWrite 2024-03-21T06:23:15.323 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio.cc:1681: Skipped 2024-03-21T06:23:15.323 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:23:15.323 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.MultiWrite (0 ms) 2024-03-21T06:23:15.324 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 16 tests from LibRadosAioEC (0 ms total) 2024-03-21T06:23:15.324 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:23:15.324 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:23:15.324 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 42 tests from 2 test suites ran. (77158 ms total) 2024-03-21T06:23:15.324 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 26 tests. 2024-03-21T06:23:15.324 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] 16 tests, listed below: 2024-03-21T06:23:15.324 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWrite 2024-03-21T06:23:15.324 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.WaitForComplete 2024-03-21T06:23:15.325 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip 2024-03-21T06:23:15.325 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip2 2024-03-21T06:23:15.325 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppend 2024-03-21T06:23:15.325 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.IsComplete 2024-03-21T06:23:15.325 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.IsSafe 2024-03-21T06:23:15.325 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValue 2024-03-21T06:23:15.325 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.Flush 2024-03-21T06:23:15.325 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsync 2024-03-21T06:23:15.325 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFull 2024-03-21T06:23:15.325 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStat 2024-03-21T06:23:15.325 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatNS 2024-03-21T06:23:15.325 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.StatRemove 2024-03-21T06:23:15.325 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClass 2024-03-21T06:23:15.325 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.MultiWrite 2024-03-21T06:23:15.326 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:23:15.326 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:23:15.326 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_aio_pp 2024-03-21T06:23:15.338 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:23:15.339 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 57 tests from 4 test suites. 2024-03-21T06:23:15.339 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:23:15.339 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 32 tests from LibRadosAio 2024-03-21T06:23:15.339 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.TooBigPP 2024-03-21T06:23:18.292 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.TooBigPP (2954 ms) 2024-03-21T06:23:18.292 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.PoolQuotaPP 2024-03-21T06:23:35.300 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.PoolQuotaPP (17007 ms) 2024-03-21T06:23:35.300 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.SimpleWritePP 2024-03-21T06:23:41.334 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.SimpleWritePP (6034 ms) 2024-03-21T06:23:41.334 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.WaitForSafePP 2024-03-21T06:23:44.345 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.WaitForSafePP (3012 ms) 2024-03-21T06:23:44.345 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RoundTripPP 2024-03-21T06:23:47.297 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RoundTripPP (2951 ms) 2024-03-21T06:23:47.297 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RoundTripPP2 2024-03-21T06:23:50.309 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RoundTripPP2 (3013 ms) 2024-03-21T06:23:50.309 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RoundTripPP3 2024-03-21T06:23:53.323 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RoundTripPP3 (3014 ms) 2024-03-21T06:23:53.323 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-03-21T06:23:56.334 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RoundTripSparseReadPP (3011 ms) 2024-03-21T06:23:56.335 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.IsCompletePP 2024-03-21T06:23:59.348 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.IsCompletePP (3014 ms) 2024-03-21T06:23:59.348 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.IsSafePP 2024-03-21T06:24:02.358 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.IsSafePP (3010 ms) 2024-03-21T06:24:02.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.ReturnValuePP 2024-03-21T06:24:05.370 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.ReturnValuePP (3011 ms) 2024-03-21T06:24:05.370 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.FlushPP 2024-03-21T06:24:08.381 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.FlushPP (3011 ms) 2024-03-21T06:24:08.381 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.FlushAsyncPP 2024-03-21T06:24:11.391 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.FlushAsyncPP (3010 ms) 2024-03-21T06:24:11.391 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-03-21T06:24:14.400 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RoundTripWriteFullPP (3009 ms) 2024-03-21T06:24:14.400 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-03-21T06:24:17.304 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RoundTripWriteFullPP2 (2905 ms) 2024-03-21T06:24:17.305 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-03-21T06:24:20.309 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RoundTripWriteSamePP (3004 ms) 2024-03-21T06:24:20.309 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-03-21T06:24:23.328 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RoundTripWriteSamePP2 (3019 ms) 2024-03-21T06:24:23.328 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.SimpleStatPPNS 2024-03-21T06:24:26.337 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.SimpleStatPPNS (3010 ms) 2024-03-21T06:24:26.337 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.SimpleStatPP 2024-03-21T06:24:29.353 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.SimpleStatPP (3015 ms) 2024-03-21T06:24:29.353 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.OperateMtime 2024-03-21T06:24:32.367 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.OperateMtime (3014 ms) 2024-03-21T06:24:32.367 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.OperateMtime2 2024-03-21T06:24:35.386 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.OperateMtime2 (3020 ms) 2024-03-21T06:24:35.386 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.StatRemovePP 2024-03-21T06:24:38.398 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.StatRemovePP (3012 ms) 2024-03-21T06:24:38.399 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.ExecuteClassPP 2024-03-21T06:24:41.410 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.ExecuteClassPP (3012 ms) 2024-03-21T06:24:41.410 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.OmapPP 2024-03-21T06:24:44.420 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.OmapPP (3010 ms) 2024-03-21T06:24:44.420 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.MultiWritePP 2024-03-21T06:24:47.310 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.MultiWritePP (2889 ms) 2024-03-21T06:24:47.310 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.AioUnlockPP 2024-03-21T06:24:50.315 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.AioUnlockPP (3006 ms) 2024-03-21T06:24:50.315 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RoundTripAppendPP 2024-03-21T06:24:53.330 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RoundTripAppendPP (3014 ms) 2024-03-21T06:24:53.330 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RacingRemovePP 2024-03-21T06:24:56.344 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RacingRemovePP (3015 ms) 2024-03-21T06:24:56.345 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-03-21T06:24:59.353 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RoundTripCmpExtPP (3008 ms) 2024-03-21T06:24:59.353 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-03-21T06:25:02.367 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.RoundTripCmpExtPP2 (3014 ms) 2024-03-21T06:25:02.367 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.PoolEIOFlag 2024-03-21T06:25:04.872 INFO:tasks.workunit.client.0.smithi096.stdout:setting pool EIO 2024-03-21T06:25:14.523 INFO:tasks.workunit.client.0.smithi096.stdout:waiting for 872 2024-03-21T06:25:15.522 INFO:tasks.workunit.client.0.smithi096.stdout:waiting for 872 2024-03-21T06:25:16.523 INFO:tasks.workunit.client.0.smithi096.stdout:waiting for 872 2024-03-21T06:25:17.523 INFO:tasks.workunit.client.0.smithi096.stdout:max_success 128, min_failed 129 2024-03-21T06:25:17.733 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAio.PoolEIOFlag (15367 ms) 2024-03-21T06:25:17.733 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAio.MultiReads 2024-03-21T06:25:17.734 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:2427: Skipped 2024-03-21T06:25:17.734 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:17.734 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAio.MultiReads (0 ms) 2024-03-21T06:25:17.734 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 32 tests from LibRadosAio (122395 ms total) 2024-03-21T06:25:17.734 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:25:17.734 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosAioPP 2024-03-21T06:25:17.734 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-03-21T06:25:20.753 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAioPP.ReadIntoBufferlist (3020 ms) 2024-03-21T06:25:20.753 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-03-21T06:25:29.349 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAioPP.XattrsRoundTripPP (8596 ms) 2024-03-21T06:25:29.349 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioPP.RmXattrPP 2024-03-21T06:25:44.333 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAioPP.RmXattrPP (14984 ms) 2024-03-21T06:25:44.333 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioPP.RemoveTestPP 2024-03-21T06:25:47.343 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosAioPP.RemoveTestPP (3010 ms) 2024-03-21T06:25:47.343 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosAioPP (29610 ms total) 2024-03-21T06:25:47.343 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:25:47.343 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibRadosIoPP 2024-03-21T06:25:47.343 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.XattrListPP 2024-03-21T06:25:50.355 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.XattrListPP (3009 ms) 2024-03-21T06:25:50.355 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibRadosIoPP (3009 ms total) 2024-03-21T06:25:50.355 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:25:50.355 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 20 tests from LibRadosAioEC 2024-03-21T06:25:50.355 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.SimpleWritePP 2024-03-21T06:25:50.356 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:1384: Skipped 2024-03-21T06:25:50.356 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.356 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWritePP (0 ms) 2024-03-21T06:25:50.356 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.WaitForSafePP 2024-03-21T06:25:50.356 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:1419: Skipped 2024-03-21T06:25:50.356 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.356 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.WaitForSafePP (0 ms) 2024-03-21T06:25:50.356 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.RoundTripPP 2024-03-21T06:25:50.356 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:1436: Skipped 2024-03-21T06:25:50.356 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.356 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP (0 ms) 2024-03-21T06:25:50.356 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.RoundTripPP2 2024-03-21T06:25:50.356 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:1467: Skipped 2024-03-21T06:25:50.357 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP2 (0 ms) 2024-03-21T06:25:50.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.RoundTripPP3 2024-03-21T06:25:50.357 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:1500: Skipped 2024-03-21T06:25:50.357 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP3 (0 ms) 2024-03-21T06:25:50.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-03-21T06:25:50.357 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:1605: Skipped 2024-03-21T06:25:50.357 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripSparseReadPP (0 ms) 2024-03-21T06:25:50.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-03-21T06:25:50.357 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:1637: Skipped 2024-03-21T06:25:50.357 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppendPP (0 ms) 2024-03-21T06:25:50.358 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.IsCompletePP 2024-03-21T06:25:50.358 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:1705: Skipped 2024-03-21T06:25:50.358 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.358 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.IsCompletePP (0 ms) 2024-03-21T06:25:50.358 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.IsSafePP 2024-03-21T06:25:50.358 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:1742: Skipped 2024-03-21T06:25:50.358 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.358 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.IsSafePP (0 ms) 2024-03-21T06:25:50.358 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.ReturnValuePP 2024-03-21T06:25:50.358 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:1780: Skipped 2024-03-21T06:25:50.358 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.358 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValuePP (0 ms) 2024-03-21T06:25:50.358 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.FlushPP 2024-03-21T06:25:50.358 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:1796: Skipped 2024-03-21T06:25:50.359 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.FlushPP (0 ms) 2024-03-21T06:25:50.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.FlushAsyncPP 2024-03-21T06:25:50.359 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:1824: Skipped 2024-03-21T06:25:50.359 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsyncPP (0 ms) 2024-03-21T06:25:50.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-03-21T06:25:50.359 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:1859: Skipped 2024-03-21T06:25:50.359 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP (0 ms) 2024-03-21T06:25:50.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-03-21T06:25:50.359 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:1904: Skipped 2024-03-21T06:25:50.359 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP2 (0 ms) 2024-03-21T06:25:50.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.SimpleStatPP 2024-03-21T06:25:50.360 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:1946: Skipped 2024-03-21T06:25:50.360 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPP (0 ms) 2024-03-21T06:25:50.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-03-21T06:25:50.360 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:1977: Skipped 2024-03-21T06:25:50.360 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPPNS (0 ms) 2024-03-21T06:25:50.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.StatRemovePP 2024-03-21T06:25:50.360 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:2008: Skipped 2024-03-21T06:25:50.360 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.StatRemovePP (0 ms) 2024-03-21T06:25:50.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.ExecuteClassPP 2024-03-21T06:25:50.360 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:2059: Skipped 2024-03-21T06:25:50.361 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClassPP (0 ms) 2024-03-21T06:25:50.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.OmapPP 2024-03-21T06:25:50.361 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:2089: Skipped 2024-03-21T06:25:50.361 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.OmapPP (0 ms) 2024-03-21T06:25:50.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosAioEC.MultiWritePP 2024-03-21T06:25:50.361 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/aio_cxx.cc:2124: Skipped 2024-03-21T06:25:50.361 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:50.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.MultiWritePP (0 ms) 2024-03-21T06:25:50.361 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 20 tests from LibRadosAioEC (0 ms total) 2024-03-21T06:25:50.361 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:25:50.361 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:25:50.361 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 57 tests from 4 test suites ran. (155014 ms total) 2024-03-21T06:25:50.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 36 tests. 2024-03-21T06:25:50.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] 21 tests, listed below: 2024-03-21T06:25:50.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAio.MultiReads 2024-03-21T06:25:50.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWritePP 2024-03-21T06:25:50.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.WaitForSafePP 2024-03-21T06:25:50.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP 2024-03-21T06:25:50.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP2 2024-03-21T06:25:50.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP3 2024-03-21T06:25:50.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripSparseReadPP 2024-03-21T06:25:50.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppendPP 2024-03-21T06:25:50.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.IsCompletePP 2024-03-21T06:25:50.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.IsSafePP 2024-03-21T06:25:50.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValuePP 2024-03-21T06:25:50.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.FlushPP 2024-03-21T06:25:50.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsyncPP 2024-03-21T06:25:50.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP 2024-03-21T06:25:50.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP2 2024-03-21T06:25:50.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPP 2024-03-21T06:25:50.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPPNS 2024-03-21T06:25:50.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.StatRemovePP 2024-03-21T06:25:50.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClassPP 2024-03-21T06:25:50.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.OmapPP 2024-03-21T06:25:50.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosAioEC.MultiWritePP 2024-03-21T06:25:50.363 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:25:50.363 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:25:50.363 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_io 2024-03-21T06:25:50.371 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:25:50.371 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 24 tests from 2 test suites. 2024-03-21T06:25:50.372 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:25:50.372 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 14 tests from LibRadosIo 2024-03-21T06:25:52.318 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIo.SimpleWrite 2024-03-21T06:25:52.508 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIo.SimpleWrite (189 ms) 2024-03-21T06:25:52.509 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIo.TooBig 2024-03-21T06:25:52.509 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIo.TooBig (0 ms) 2024-03-21T06:25:52.509 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIo.ReadTimeout 2024-03-21T06:25:52.549 INFO:tasks.workunit.client.0.smithi096.stdout:no timeout :/ 2024-03-21T06:25:52.549 INFO:tasks.workunit.client.0.smithi096.stdout:no timeout :/ 2024-03-21T06:25:52.549 INFO:tasks.workunit.client.0.smithi096.stdout:no timeout :/ 2024-03-21T06:25:52.549 INFO:tasks.workunit.client.0.smithi096.stdout:no timeout :/ 2024-03-21T06:25:52.549 INFO:tasks.workunit.client.0.smithi096.stdout:no timeout :/ 2024-03-21T06:25:52.549 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIo.ReadTimeout (42 ms) 2024-03-21T06:25:52.550 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIo.RoundTrip 2024-03-21T06:25:52.570 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIo.RoundTrip (21 ms) 2024-03-21T06:25:52.570 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIo.Checksum 2024-03-21T06:25:52.579 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIo.Checksum (9 ms) 2024-03-21T06:25:52.579 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-03-21T06:25:52.623 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIo.OverlappingWriteRoundTrip (43 ms) 2024-03-21T06:25:52.623 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIo.WriteFullRoundTrip 2024-03-21T06:25:52.637 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIo.WriteFullRoundTrip (15 ms) 2024-03-21T06:25:52.637 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIo.AppendRoundTrip 2024-03-21T06:25:52.650 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIo.AppendRoundTrip (13 ms) 2024-03-21T06:25:52.651 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIo.ZeroLenZero 2024-03-21T06:25:52.655 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIo.ZeroLenZero (5 ms) 2024-03-21T06:25:52.655 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIo.TruncTest 2024-03-21T06:25:52.669 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIo.TruncTest (14 ms) 2024-03-21T06:25:52.669 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIo.RemoveTest 2024-03-21T06:25:52.683 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIo.RemoveTest (14 ms) 2024-03-21T06:25:52.683 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIo.XattrsRoundTrip 2024-03-21T06:25:52.699 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIo.XattrsRoundTrip (16 ms) 2024-03-21T06:25:52.699 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIo.RmXattr 2024-03-21T06:25:52.738 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIo.RmXattr (38 ms) 2024-03-21T06:25:52.738 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIo.XattrIter 2024-03-21T06:25:52.757 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIo.XattrIter (20 ms) 2024-03-21T06:25:53.368 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 14 tests from LibRadosIo (439 ms total) 2024-03-21T06:25:53.368 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:25:53.368 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 10 tests from LibRadosIoEC 2024-03-21T06:25:53.368 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:168: Skipped 2024-03-21T06:25:53.368 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoEC.SimpleWrite 2024-03-21T06:25:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:25:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:25:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.SimpleWrite (0 ms) 2024-03-21T06:25:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoEC.RoundTrip 2024-03-21T06:25:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:25:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:25:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.RoundTrip (0 ms) 2024-03-21T06:25:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-03-21T06:25:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:25:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.370 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:25:53.370 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.370 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.OverlappingWriteRoundTrip (0 ms) 2024-03-21T06:25:53.370 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-03-21T06:25:53.370 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:25:53.370 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.370 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:25:53.370 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.370 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.WriteFullRoundTrip (0 ms) 2024-03-21T06:25:53.370 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoEC.AppendRoundTrip 2024-03-21T06:25:53.370 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:25:53.370 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.370 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:25:53.371 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.371 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.AppendRoundTrip (0 ms) 2024-03-21T06:25:53.371 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoEC.TruncTest 2024-03-21T06:25:53.371 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:25:53.371 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.371 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:25:53.371 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.371 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.TruncTest (0 ms) 2024-03-21T06:25:53.371 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoEC.RemoveTest 2024-03-21T06:25:53.371 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:25:53.371 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.371 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:25:53.371 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.371 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.RemoveTest (0 ms) 2024-03-21T06:25:53.372 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-03-21T06:25:53.372 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:25:53.372 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.372 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:25:53.372 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.372 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.XattrsRoundTrip (0 ms) 2024-03-21T06:25:53.372 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoEC.RmXattr 2024-03-21T06:25:53.372 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:25:53.372 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.372 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:25:53.372 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.372 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.RmXattr (0 ms) 2024-03-21T06:25:53.372 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoEC.XattrIter 2024-03-21T06:25:53.372 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:25:53.372 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.373 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:25:53.373 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.373 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.XattrIter (0 ms) 2024-03-21T06:25:53.373 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:176: Skipped 2024-03-21T06:25:53.373 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:53.373 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 10 tests from LibRadosIoEC (0 ms total) 2024-03-21T06:25:53.373 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:25:53.373 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:25:53.373 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 24 tests from 2 test suites ran. (2995 ms total) 2024-03-21T06:25:53.373 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 14 tests. 2024-03-21T06:25:53.373 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] 10 tests, listed below: 2024-03-21T06:25:53.373 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.SimpleWrite 2024-03-21T06:25:53.373 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.RoundTrip 2024-03-21T06:25:53.373 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-03-21T06:25:53.373 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.WriteFullRoundTrip 2024-03-21T06:25:53.374 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.AppendRoundTrip 2024-03-21T06:25:53.374 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.TruncTest 2024-03-21T06:25:53.374 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.RemoveTest 2024-03-21T06:25:53.374 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.XattrsRoundTrip 2024-03-21T06:25:53.374 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.RmXattr 2024-03-21T06:25:53.374 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoEC.XattrIter 2024-03-21T06:25:53.374 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:25:53.375 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:25:53.375 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_io_pp 2024-03-21T06:25:53.388 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:25:53.388 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 37 tests from 2 test suites. 2024-03-21T06:25:53.388 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:25:53.388 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 20 tests from LibRadosIoPP 2024-03-21T06:25:53.388 INFO:tasks.workunit.client.0.smithi096.stdout:seed 33876 2024-03-21T06:25:55.370 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.TooBigPP 2024-03-21T06:25:55.371 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-03-21T06:25:55.371 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.SimpleWritePP 2024-03-21T06:25:55.590 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.SimpleWritePP (220 ms) 2024-03-21T06:25:55.591 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.ReadOpPP 2024-03-21T06:25:55.621 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.ReadOpPP (30 ms) 2024-03-21T06:25:55.621 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.SparseReadOpPP 2024-03-21T06:25:55.631 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.SparseReadOpPP (11 ms) 2024-03-21T06:25:55.631 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.RoundTripPP 2024-03-21T06:25:55.639 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.RoundTripPP (7 ms) 2024-03-21T06:25:55.639 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.RoundTripPP2 2024-03-21T06:25:55.647 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.RoundTripPP2 (9 ms) 2024-03-21T06:25:55.647 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.Checksum 2024-03-21T06:25:55.657 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.Checksum (10 ms) 2024-03-21T06:25:55.657 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-03-21T06:25:55.666 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.ReadIntoBufferlist (9 ms) 2024-03-21T06:25:55.667 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-03-21T06:25:55.684 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (18 ms) 2024-03-21T06:25:55.684 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-03-21T06:25:55.698 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.WriteFullRoundTripPP (14 ms) 2024-03-21T06:25:55.698 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-03-21T06:25:55.707 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (9 ms) 2024-03-21T06:25:55.707 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-03-21T06:25:55.720 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.AppendRoundTripPP (13 ms) 2024-03-21T06:25:55.720 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.TruncTestPP 2024-03-21T06:25:55.734 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.TruncTestPP (14 ms) 2024-03-21T06:25:55.734 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.RemoveTestPP 2024-03-21T06:25:55.747 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.RemoveTestPP (13 ms) 2024-03-21T06:25:55.747 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-03-21T06:25:55.763 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.XattrsRoundTripPP (16 ms) 2024-03-21T06:25:55.763 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.RmXattrPP 2024-03-21T06:25:55.801 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.RmXattrPP (38 ms) 2024-03-21T06:25:55.801 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.XattrListPP 2024-03-21T06:25:55.819 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.XattrListPP (18 ms) 2024-03-21T06:25:55.819 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.CmpExtPP 2024-03-21T06:25:55.837 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.CmpExtPP (18 ms) 2024-03-21T06:25:55.837 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-03-21T06:25:55.845 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.CmpExtDNEPP (8 ms) 2024-03-21T06:25:55.845 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-03-21T06:25:55.858 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosIoPP.CmpExtMismatchPP (13 ms) 2024-03-21T06:25:56.376 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 20 tests from LibRadosIoPP (488 ms total) 2024-03-21T06:25:56.376 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:25:56.376 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 17 tests from LibRadosIoECPP 2024-03-21T06:25:56.376 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-21T06:25:56.376 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.376 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoECPP.SimpleWritePP 2024-03-21T06:25:56.376 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:25:56.376 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.376 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:25:56.377 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.377 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.SimpleWritePP (0 ms) 2024-03-21T06:25:56.377 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoECPP.ReadOpPP 2024-03-21T06:25:56.377 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:25:56.377 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.377 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:25:56.377 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.377 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.ReadOpPP (0 ms) 2024-03-21T06:25:56.377 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-03-21T06:25:56.377 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:25:56.377 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.377 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:25:56.377 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.377 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.SparseReadOpPP (0 ms) 2024-03-21T06:25:56.378 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoECPP.RoundTripPP 2024-03-21T06:25:56.378 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:25:56.378 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.378 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:25:56.378 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.378 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP (0 ms) 2024-03-21T06:25:56.378 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoECPP.RoundTripPP2 2024-03-21T06:25:56.378 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:25:56.378 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.378 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:25:56.378 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.378 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP2 (0 ms) 2024-03-21T06:25:56.378 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-03-21T06:25:56.378 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:25:56.379 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.379 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:25:56.379 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.379 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.OverlappingWriteRoundTripPP (0 ms) 2024-03-21T06:25:56.379 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-03-21T06:25:56.379 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:25:56.379 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.379 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:25:56.379 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.379 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP (0 ms) 2024-03-21T06:25:56.379 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-03-21T06:25:56.379 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:25:56.379 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.379 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:25:56.379 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.380 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP2 (0 ms) 2024-03-21T06:25:56.380 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-03-21T06:25:56.380 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:25:56.380 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.380 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:25:56.380 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.380 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.AppendRoundTripPP (0 ms) 2024-03-21T06:25:56.380 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoECPP.TruncTestPP 2024-03-21T06:25:56.380 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:25:56.380 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.380 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:25:56.380 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.380 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.TruncTestPP (0 ms) 2024-03-21T06:25:56.380 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoECPP.RemoveTestPP 2024-03-21T06:25:56.381 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:25:56.381 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.381 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:25:56.381 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.381 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.RemoveTestPP (0 ms) 2024-03-21T06:25:56.381 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-03-21T06:25:56.381 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:25:56.381 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.381 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:25:56.381 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.381 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.XattrsRoundTripPP (0 ms) 2024-03-21T06:25:56.381 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoECPP.RmXattrPP 2024-03-21T06:25:56.381 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:25:56.381 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.382 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:25:56.382 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.382 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.RmXattrPP (0 ms) 2024-03-21T06:25:56.382 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoECPP.XattrListPP 2024-03-21T06:25:56.382 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:25:56.382 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.382 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:25:56.382 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.382 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.XattrListPP (0 ms) 2024-03-21T06:25:56.382 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoECPP.CmpExtPP 2024-03-21T06:25:56.382 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:25:56.382 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.382 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:25:56.383 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.383 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtPP (0 ms) 2024-03-21T06:25:56.383 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-03-21T06:25:56.383 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:25:56.383 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.383 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:25:56.383 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.383 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtDNEPP (0 ms) 2024-03-21T06:25:56.383 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-03-21T06:25:56.383 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:25:56.383 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.383 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:25:56.383 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.383 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtMismatchPP (0 ms) 2024-03-21T06:25:56.383 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-21T06:25:56.384 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:25:56.384 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 17 tests from LibRadosIoECPP (0 ms total) 2024-03-21T06:25:56.384 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:25:56.384 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:25:56.384 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 37 tests from 2 test suites ran. (2984 ms total) 2024-03-21T06:25:56.384 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 20 tests. 2024-03-21T06:25:56.384 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] 17 tests, listed below: 2024-03-21T06:25:56.384 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.SimpleWritePP 2024-03-21T06:25:56.384 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.ReadOpPP 2024-03-21T06:25:56.384 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.SparseReadOpPP 2024-03-21T06:25:56.384 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP 2024-03-21T06:25:56.384 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP2 2024-03-21T06:25:56.384 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-03-21T06:25:56.384 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP 2024-03-21T06:25:56.385 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-03-21T06:25:56.385 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.AppendRoundTripPP 2024-03-21T06:25:56.385 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.TruncTestPP 2024-03-21T06:25:56.385 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.RemoveTestPP 2024-03-21T06:25:56.385 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.XattrsRoundTripPP 2024-03-21T06:25:56.385 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.RmXattrPP 2024-03-21T06:25:56.385 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.XattrListPP 2024-03-21T06:25:56.385 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtPP 2024-03-21T06:25:56.385 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtDNEPP 2024-03-21T06:25:56.385 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtMismatchPP 2024-03-21T06:25:56.386 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:25:56.386 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:25:56.386 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_asio 2024-03-21T06:25:56.400 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.400+0000 7f3ee9c74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:25:56.404 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.404+0000 7f3ee9c74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:25:56.404 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.404+0000 7f3ee9c74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:25:56.404 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 12 tests from 1 test suite. 2024-03-21T06:25:56.404 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:25:56.405 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 12 tests from AsioRados 2024-03-21T06:25:56.406 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.404+0000 7f3ee9c74640 1 Processor -- start 2024-03-21T06:25:56.406 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.404+0000 7f3ee9c74640 1 -- start start 2024-03-21T06:25:56.406 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.405+0000 7f3ee9c74640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56282e5af0a0 0x56282e5b0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:25:56.406 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.405+0000 7f3ee9c74640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x56282e4b3ea0 con 0x56282e5af0a0 2024-03-21T06:25:56.406 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.405+0000 7f3ee2049640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56282e5af0a0 0x56282e5b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:25:56.406 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.405+0000 7f3ee2049640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56282e5af0a0 0x56282e5b0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51026/0 (socket says 172.21.15.96:51026) 2024-03-21T06:25:56.406 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.405+0000 7f3ee2049640 1 -- 172.21.15.96:0/847803850 learned_addr learned my addr 172.21.15.96:0/847803850 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:25:56.407 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.405+0000 7f3ee2049640 1 -- 172.21.15.96:0/847803850 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56282e456a30 con 0x56282e5af0a0 2024-03-21T06:25:56.407 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.405+0000 7f3ee2049640 1 --2- 172.21.15.96:0/847803850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56282e5af0a0 0x56282e5b0290 crc :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c1dea5aafacb52 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:25:56.407 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.406+0000 7f3ee0846640 1 -- 172.21.15.96:0/847803850 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3ed80061f0 con 0x56282e5af0a0 2024-03-21T06:25:56.407 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.406+0000 7f3ee0846640 1 -- 172.21.15.96:0/847803850 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3ed8001940 con 0x56282e5af0a0 2024-03-21T06:25:56.407 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.406+0000 7f3ee0846640 1 -- 172.21.15.96:0/847803850 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3ed8001ab0 con 0x56282e5af0a0 2024-03-21T06:25:56.409 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.406+0000 7f3ee9c74640 1 -- 172.21.15.96:0/847803850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56282e5af0a0 msgr2=0x56282e5b0290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:25:56.410 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.406+0000 7f3ee9c74640 1 --2- 172.21.15.96:0/847803850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56282e5af0a0 0x56282e5b0290 crc :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:25:56.410 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.406+0000 7f3ee9c74640 1 -- 172.21.15.96:0/847803850 shutdown_connections 2024-03-21T06:25:56.410 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.406+0000 7f3ee9c74640 1 --2- 172.21.15.96:0/847803850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56282e5af0a0 0x56282e5b0290 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:25:56.410 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.406+0000 7f3ee9c74640 1 -- 172.21.15.96:0/847803850 >> 172.21.15.96:0/847803850 conn(0x56282e428e00 msgr2=0x56282e514750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:25:56.410 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.406+0000 7f3ee9c74640 1 -- 172.21.15.96:0/847803850 shutdown_connections 2024-03-21T06:25:56.410 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.406+0000 7f3ee9c74640 1 -- 172.21.15.96:0/847803850 wait complete. 2024-03-21T06:25:56.410 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.407+0000 7f3ee9c74640 1 Processor -- start 2024-03-21T06:25:56.410 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.407+0000 7f3ee9c74640 1 -- start start 2024-03-21T06:25:56.410 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.408+0000 7f3ee9c74640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56282e5af0a0 0x56282e597a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:25:56.410 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.408+0000 7f3ee9c74640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x56282e454550 con 0x56282e5af0a0 2024-03-21T06:25:56.410 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.408+0000 7f3ee2049640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56282e5af0a0 0x56282e597a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:25:56.410 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.408+0000 7f3ee2049640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56282e5af0a0 0x56282e597a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51038/0 (socket says 172.21.15.96:51038) 2024-03-21T06:25:56.411 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.408+0000 7f3ee2049640 1 -- 172.21.15.96:0/3658899151 learned_addr learned my addr 172.21.15.96:0/3658899151 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:25:56.411 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.408+0000 7f3ee2049640 1 -- 172.21.15.96:0/3658899151 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56282e449680 con 0x56282e5af0a0 2024-03-21T06:25:56.411 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.408+0000 7f3ee2049640 1 --2- 172.21.15.96:0/3658899151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56282e5af0a0 0x56282e597a30 crc :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:25:56.411 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.408+0000 7f3ed37fe640 1 -- 172.21.15.96:0/3658899151 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3ed8011c20 con 0x56282e5af0a0 2024-03-21T06:25:56.411 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.408+0000 7f3ed37fe640 1 -- 172.21.15.96:0/3658899151 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3ed8011d80 con 0x56282e5af0a0 2024-03-21T06:25:56.411 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.408+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56282e4490e0 con 0x56282e5af0a0 2024-03-21T06:25:56.411 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.408+0000 7f3ed37fe640 1 -- 172.21.15.96:0/3658899151 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3ed8012460 con 0x56282e5af0a0 2024-03-21T06:25:56.411 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.408+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56282e59a680 con 0x56282e5af0a0 2024-03-21T06:25:56.411 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.409+0000 7f3ed37fe640 1 -- 172.21.15.96:0/3658899151 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f3ed8012650 con 0x56282e5af0a0 2024-03-21T06:25:56.411 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.409+0000 7f3ed37fe640 1 --2- 172.21.15.96:0/3658899151 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3ec0036b80 0x7f3ec0039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:25:56.411 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.409+0000 7f3ed37fe640 1 -- 172.21.15.96:0/3658899151 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 2672+0+0 (crc 0 0 0) 0x7f3ed8047730 con 0x56282e5af0a0 2024-03-21T06:25:56.411 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.409+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x56282e4aa620 con 0x56282e5af0a0 2024-03-21T06:25:56.411 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.409+0000 7f3ed37fe640 1 -- 172.21.15.96:0/3658899151 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=234) v2 ==== 24+0+0 (crc 0 0 0) 0x7f3ed80037d0 con 0x56282e5af0a0 2024-03-21T06:25:56.412 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.409+0000 7f3ee1848640 1 --2- 172.21.15.96:0/3658899151 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3ec0036b80 0x7f3ec0039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:25:56.412 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.409+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- pool_op(create pool 0 tid 1 name ceph_test_rados_api_asio v0) v4 -- 0x56282e4bbda0 con 0x56282e5af0a0 2024-03-21T06:25:56.412 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:56.410+0000 7f3ee1848640 1 --2- 172.21.15.96:0/3658899151 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3ec0036b80 0x7f3ec0039090 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:25:57.378 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:57.377+0000 7f3ed37fe640 1 -- 172.21.15.96:0/3658899151 <== mon.0 v2:172.21.15.96:3300/0 7 ==== pool_op_reply(tid 1 (0) Success v235) v1 ==== 43+0+0 (crc 0 0 0) 0x7f3ed8003450 con 0x56282e5af0a0 2024-03-21T06:25:57.378 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:57.377+0000 7f3ed37fe640 1 -- 172.21.15.96:0/3658899151 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=235}) v3 -- 0x7f3ec003b500 con 0x56282e5af0a0 2024-03-21T06:25:57.378 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:57.377+0000 7f3ed37fe640 1 -- 172.21.15.96:0/3658899151 <== mon.0 v2:172.21.15.96:3300/0 8 ==== osd_map(235..235 src has 184..235) v4 ==== 651+0+0 (crc 0 0 0) 0x7f3ed800d3a0 con 0x56282e5af0a0 2024-03-21T06:25:57.378 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:57.377+0000 7f3ee9c74640 1 --2- 172.21.15.96:0/3658899151 >> v2:172.21.15.96:6805/13988474 conn(0x56282e5afbc0 0x56282e598480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:25:57.378 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:57.377+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:2 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e64c600 con 0x56282e5afbc0 2024-03-21T06:25:57.379 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:57.379+0000 7f3ee284a640 1 --2- 172.21.15.96:0/3658899151 >> v2:172.21.15.96:6805/13988474 conn(0x56282e5afbc0 0x56282e598480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:25:57.383 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:57.383+0000 7f3ee284a640 1 --2- 172.21.15.96:0/3658899151 >> v2:172.21.15.96:6805/13988474 conn(0x56282e5afbc0 0x56282e598480 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:25:58.524 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.524+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 1 ==== osd_op_reply(2 exist [writefull 0~5] v236'1 uv1 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.524 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] AsioRados.AsyncReadCallback 2024-03-21T06:25:58.525 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.524+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:3 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e6495e0 con 0x56282e5afbc0 2024-03-21T06:25:58.525 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.524+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:4 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e649e80 con 0x56282e5afbc0 2024-03-21T06:25:58.533 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.533+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 2 ==== osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.534 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.533+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 3 ==== osd_op_reply(3 exist [read 0~5 out=5b] v236'1 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.534 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] AsioRados.AsyncReadCallback (9 ms) 2024-03-21T06:25:58.534 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] AsioRados.AsyncReadFuture 2024-03-21T06:25:58.534 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.533+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:5 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e64a210 con 0x56282e5afbc0 2024-03-21T06:25:58.534 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.533+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:6 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e64c960 con 0x56282e5afbc0 2024-03-21T06:25:58.540 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.540+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 4 ==== osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.541 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.540+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 5 ==== osd_op_reply(5 exist [read 0~5 out=5b] v236'1 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.541 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] AsioRados.AsyncReadFuture (7 ms) 2024-03-21T06:25:58.541 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] AsioRados.AsyncReadYield 2024-03-21T06:25:58.541 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.540+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:7 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e59b230 con 0x56282e5afbc0 2024-03-21T06:25:58.541 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.540+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:8 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e68d800 con 0x56282e5afbc0 2024-03-21T06:25:58.545 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.544+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 6 ==== osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.545 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.545+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 7 ==== osd_op_reply(7 exist [read 0~5 out=5b] v236'1 uv1 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.545 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] AsioRados.AsyncReadYield (5 ms) 2024-03-21T06:25:58.545 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] AsioRados.AsyncWriteCallback 2024-03-21T06:25:58.545 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.545+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:9 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e64dcd0 con 0x56282e5afbc0 2024-03-21T06:25:58.551 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.551+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 8 ==== osd_op_reply(9 exist [write 0~5] v236'2 uv2 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.551 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] AsioRados.AsyncWriteCallback (6 ms) 2024-03-21T06:25:58.551 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] AsioRados.AsyncWriteFuture 2024-03-21T06:25:58.551 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.551+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:10 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e64dcd0 con 0x56282e5afbc0 2024-03-21T06:25:58.557 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.557+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 9 ==== osd_op_reply(10 exist [write 0~5] v236'3 uv3 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.557 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] AsioRados.AsyncWriteFuture (6 ms) 2024-03-21T06:25:58.557 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] AsioRados.AsyncWriteYield 2024-03-21T06:25:58.557 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.557+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:11 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e6701e0 con 0x56282e5afbc0 2024-03-21T06:25:58.564 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.564+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 10 ==== osd_op_reply(11 exist [write 0~5] v236'4 uv4 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.565 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] AsioRados.AsyncWriteYield (7 ms) 2024-03-21T06:25:58.565 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] AsioRados.AsyncReadOperationCallback 2024-03-21T06:25:58.565 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.564+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:12 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e64ccc0 con 0x56282e5afbc0 2024-03-21T06:25:58.565 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.564+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:13 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e64d620 con 0x56282e5afbc0 2024-03-21T06:25:58.568 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.568+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 11 ==== osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.568 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.568+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 12 ==== osd_op_reply(12 exist [read 0~5 out=5b] v236'4 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.569 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] AsioRados.AsyncReadOperationCallback (4 ms) 2024-03-21T06:25:58.569 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] AsioRados.AsyncReadOperationFuture 2024-03-21T06:25:58.569 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.568+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:14 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e64d170 con 0x56282e5afbc0 2024-03-21T06:25:58.569 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.568+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:15 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e670860 con 0x56282e5afbc0 2024-03-21T06:25:58.571 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.571+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 13 ==== osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.571 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.571+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 14 ==== osd_op_reply(14 exist [read 0~5 out=5b] v236'4 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.571 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] AsioRados.AsyncReadOperationFuture (3 ms) 2024-03-21T06:25:58.571 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] AsioRados.AsyncReadOperationYield 2024-03-21T06:25:58.572 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.571+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:16 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e64d6a0 con 0x56282e5afbc0 2024-03-21T06:25:58.572 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.571+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:17 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e670020 con 0x56282e5afbc0 2024-03-21T06:25:58.574 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.574+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 15 ==== osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.574 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.574+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 16 ==== osd_op_reply(16 exist [read 0~5 out=5b] v236'4 uv4 ondisk = 0) v8 ==== 149+0+5 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.574 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] AsioRados.AsyncReadOperationYield (3 ms) 2024-03-21T06:25:58.574 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] AsioRados.AsyncWriteOperationCallback 2024-03-21T06:25:58.575 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.574+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:18 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e64d6a0 con 0x56282e5afbc0 2024-03-21T06:25:58.579 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] AsioRados.AsyncWriteOperationCallback (5 ms) 2024-03-21T06:25:58.579 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] AsioRados.AsyncWriteOperationFuture 2024-03-21T06:25:58.580 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.579+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 17 ==== osd_op_reply(18 exist [writefull 0~5] v236'5 uv5 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.580 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.579+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:19 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e64d6a0 con 0x56282e5afbc0 2024-03-21T06:25:58.585 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.585+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 18 ==== osd_op_reply(19 exist [writefull 0~5] v236'6 uv6 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.586 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] AsioRados.AsyncWriteOperationFuture (6 ms) 2024-03-21T06:25:58.586 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] AsioRados.AsyncWriteOperationYield 2024-03-21T06:25:58.586 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.585+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:20 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235) v9 -- 0x56282e64d6a0 con 0x56282e5afbc0 2024-03-21T06:25:58.591 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.590+0000 7f3ee284a640 1 -- 172.21.15.96:0/3658899151 <== osd.1 v2:172.21.15.96:6805/13988474 19 ==== osd_op_reply(20 exist [writefull 0~5] v236'7 uv7 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f3ed400c430 con 0x56282e5afbc0 2024-03-21T06:25:58.591 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] AsioRados.AsyncWriteOperationYield (6 ms) 2024-03-21T06:25:58.592 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.591+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 >> v2:172.21.15.96:6805/13988474 conn(0x56282e5afbc0 msgr2=0x56282e598480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:25:58.592 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.591+0000 7f3ee9c74640 1 --2- 172.21.15.96:0/3658899151 >> v2:172.21.15.96:6805/13988474 conn(0x56282e5afbc0 0x56282e598480 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:25:58.592 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.591+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3ec0036b80 msgr2=0x7f3ec0039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:25:58.592 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.591+0000 7f3ee9c74640 1 --2- 172.21.15.96:0/3658899151 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3ec0036b80 0x7f3ec0039090 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:25:58.592 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.591+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56282e5af0a0 msgr2=0x56282e597a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:25:58.592 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.591+0000 7f3ee9c74640 1 --2- 172.21.15.96:0/3658899151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56282e5af0a0 0x56282e597a30 crc :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:25:58.592 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.591+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 shutdown_connections 2024-03-21T06:25:58.592 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.591+0000 7f3ee9c74640 1 --2- 172.21.15.96:0/3658899151 >> v2:172.21.15.96:6805/13988474 conn(0x56282e5afbc0 0x56282e598480 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:25:58.593 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.591+0000 7f3ee9c74640 1 --2- 172.21.15.96:0/3658899151 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3ec0036b80 0x7f3ec0039090 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:25:58.593 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.591+0000 7f3ee9c74640 1 --2- 172.21.15.96:0/3658899151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56282e5af0a0 0x56282e597a30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:25:58.593 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.591+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 >> 172.21.15.96:0/3658899151 conn(0x56282e428e00 msgr2=0x56282e514750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:25:58.593 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.591+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 shutdown_connections 2024-03-21T06:25:58.593 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.591+0000 7f3ee9c74640 1 -- 172.21.15.96:0/3658899151 wait complete. 2024-03-21T06:25:58.593 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 12 tests from AsioRados (67 ms total) 2024-03-21T06:25:58.593 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:25:58.593 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:25:58.593 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 12 tests from 1 test suite ran. (2187 ms total) 2024-03-21T06:25:58.594 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 12 tests. 2024-03-21T06:25:58.594 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:25:58.594 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:25:58.594 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_list 2024-03-21T06:25:58.618 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.618+0000 7f15a2236940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:25:58.619 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.618+0000 7f15a2236940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:25:58.619 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:25:58.618+0000 7f15a2236940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:25:58.619 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 11 tests from 3 test suites. 2024-03-21T06:25:58.619 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:25:58.619 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 7 tests from LibRadosList 2024-03-21T06:26:00.392 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosList.ListObjects 2024-03-21T06:26:00.744 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosList.ListObjects (352 ms) 2024-03-21T06:26:00.744 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosList.ListObjectsZeroInName 2024-03-21T06:26:00.854 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosList.ListObjectsZeroInName (110 ms) 2024-03-21T06:26:00.855 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosList.ListObjectsNS 2024-03-21T06:26:00.904 INFO:tasks.workunit.client.0.smithi096.stdout:myset foo1,foo2,foo3 2024-03-21T06:26:00.954 INFO:tasks.workunit.client.0.smithi096.stdout:foo1 2024-03-21T06:26:00.955 INFO:tasks.workunit.client.0.smithi096.stdout:foo2 2024-03-21T06:26:00.955 INFO:tasks.workunit.client.0.smithi096.stdout:foo3 2024-03-21T06:26:00.955 INFO:tasks.workunit.client.0.smithi096.stdout:myset foo1,foo4,foo5 2024-03-21T06:26:01.004 INFO:tasks.workunit.client.0.smithi096.stdout:foo4 2024-03-21T06:26:01.004 INFO:tasks.workunit.client.0.smithi096.stdout:foo5 2024-03-21T06:26:01.004 INFO:tasks.workunit.client.0.smithi096.stdout:foo1 2024-03-21T06:26:01.004 INFO:tasks.workunit.client.0.smithi096.stdout:myset foo6,foo7 2024-03-21T06:26:01.053 INFO:tasks.workunit.client.0.smithi096.stdout:foo7 2024-03-21T06:26:01.054 INFO:tasks.workunit.client.0.smithi096.stdout:foo6 2024-03-21T06:26:01.054 INFO:tasks.workunit.client.0.smithi096.stdout:myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-03-21T06:26:01.104 INFO:tasks.workunit.client.0.smithi096.stdout:ns1:foo4 2024-03-21T06:26:01.104 INFO:tasks.workunit.client.0.smithi096.stdout:ns1:foo5 2024-03-21T06:26:01.104 INFO:tasks.workunit.client.0.smithi096.stdout:ns2:foo7 2024-03-21T06:26:01.104 INFO:tasks.workunit.client.0.smithi096.stdout:ns2:foo6 2024-03-21T06:26:01.104 INFO:tasks.workunit.client.0.smithi096.stdout:ns1:foo1 2024-03-21T06:26:01.104 INFO:tasks.workunit.client.0.smithi096.stdout::foo1 2024-03-21T06:26:01.104 INFO:tasks.workunit.client.0.smithi096.stdout::foo2 2024-03-21T06:26:01.104 INFO:tasks.workunit.client.0.smithi096.stdout::foo3 2024-03-21T06:26:01.196 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosList.ListObjectsNS (341 ms) 2024-03-21T06:26:01.196 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosList.ListObjectsStart 2024-03-21T06:26:01.344 INFO:tasks.workunit.client.0.smithi096.stdout:1 0 2024-03-21T06:26:01.344 INFO:tasks.workunit.client.0.smithi096.stdout:10 0 2024-03-21T06:26:01.344 INFO:tasks.workunit.client.0.smithi096.stdout:13 0 2024-03-21T06:26:01.344 INFO:tasks.workunit.client.0.smithi096.stdout:7 0 2024-03-21T06:26:01.344 INFO:tasks.workunit.client.0.smithi096.stdout:14 0 2024-03-21T06:26:01.344 INFO:tasks.workunit.client.0.smithi096.stdout:0 0 2024-03-21T06:26:01.344 INFO:tasks.workunit.client.0.smithi096.stdout:15 0 2024-03-21T06:26:01.345 INFO:tasks.workunit.client.0.smithi096.stdout:11 0 2024-03-21T06:26:01.345 INFO:tasks.workunit.client.0.smithi096.stdout:5 0 2024-03-21T06:26:01.345 INFO:tasks.workunit.client.0.smithi096.stdout:8 0 2024-03-21T06:26:01.345 INFO:tasks.workunit.client.0.smithi096.stdout:6 0 2024-03-21T06:26:01.345 INFO:tasks.workunit.client.0.smithi096.stdout:3 0 2024-03-21T06:26:01.345 INFO:tasks.workunit.client.0.smithi096.stdout:4 0 2024-03-21T06:26:01.345 INFO:tasks.workunit.client.0.smithi096.stdout:12 0 2024-03-21T06:26:01.345 INFO:tasks.workunit.client.0.smithi096.stdout:9 0 2024-03-21T06:26:01.345 INFO:tasks.workunit.client.0.smithi096.stdout:2 0 2024-03-21T06:26:01.394 INFO:tasks.workunit.client.0.smithi096.stdout:have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-03-21T06:26:01.563 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosList.ListObjectsStart (367 ms) 2024-03-21T06:26:01.563 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosList.ListObjectsCursor 2024-03-21T06:26:01.654 INFO:tasks.workunit.client.0.smithi096.stdout:x cursor=MIN 2024-03-21T06:26:01.706 INFO:tasks.workunit.client.0.smithi096.stdout:> oid=1 cursor=76:02547ec2:::1:head 2024-03-21T06:26:01.706 INFO:tasks.workunit.client.0.smithi096.stdout:> oid=10 cursor=76:52ea6a34:::10:head 2024-03-21T06:26:01.706 INFO:tasks.workunit.client.0.smithi096.stdout:> oid=13 cursor=76:566253c9:::13:head 2024-03-21T06:26:01.706 INFO:tasks.workunit.client.0.smithi096.stdout:> oid=7 cursor=76:5c6b0b28:::7:head 2024-03-21T06:26:01.706 INFO:tasks.workunit.client.0.smithi096.stdout:> oid=14 cursor=76:62a1935d:::14:head 2024-03-21T06:26:01.706 INFO:tasks.workunit.client.0.smithi096.stdout:> oid=0 cursor=76:6cac518f:::0:head 2024-03-21T06:26:01.706 INFO:tasks.workunit.client.0.smithi096.stdout:> oid=15 cursor=76:863748b0:::15:head 2024-03-21T06:26:01.707 INFO:tasks.workunit.client.0.smithi096.stdout:> oid=11 cursor=76:89d3ae78:::11:head 2024-03-21T06:26:01.707 INFO:tasks.workunit.client.0.smithi096.stdout:> oid=5 cursor=76:b29083e3:::5:head 2024-03-21T06:26:01.707 INFO:tasks.workunit.client.0.smithi096.stdout:> oid=8 cursor=76:bd63b0f1:::8:head 2024-03-21T06:26:01.707 INFO:tasks.workunit.client.0.smithi096.stdout:> oid=6 cursor=76:c4fdafeb:::6:head 2024-03-21T06:26:01.707 INFO:tasks.workunit.client.0.smithi096.stdout:> oid=3 cursor=76:cfc208b3:::3:head 2024-03-21T06:26:01.707 INFO:tasks.workunit.client.0.smithi096.stdout:> oid=4 cursor=76:d83876eb:::4:head 2024-03-21T06:26:01.707 INFO:tasks.workunit.client.0.smithi096.stdout:> oid=12 cursor=76:de5d7c5f:::12:head 2024-03-21T06:26:01.707 INFO:tasks.workunit.client.0.smithi096.stdout:> oid=9 cursor=76:e960b815:::9:head 2024-03-21T06:26:01.707 INFO:tasks.workunit.client.0.smithi096.stdout:> oid=2 cursor=76:f905c69b:::2:head 2024-03-21T06:26:01.758 INFO:tasks.workunit.client.0.smithi096.stdout:FIRST> seek to MIN oid=1 2024-03-21T06:26:01.808 INFO:tasks.workunit.client.0.smithi096.stdout:: oid=1 cursor=76:02547ec2:::1:head 2024-03-21T06:26:01.808 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:02547ec2:::1:head 2024-03-21T06:26:01.859 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:02547ec2:::1:head -> 1 2024-03-21T06:26:01.859 INFO:tasks.workunit.client.0.smithi096.stdout:: oid=10 cursor=76:52ea6a34:::10:head 2024-03-21T06:26:01.859 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:52ea6a34:::10:head 2024-03-21T06:26:01.893 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:52ea6a34:::10:head -> 10 2024-03-21T06:26:01.893 INFO:tasks.workunit.client.0.smithi096.stdout:: oid=13 cursor=76:566253c9:::13:head 2024-03-21T06:26:01.894 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:566253c9:::13:head 2024-03-21T06:26:01.928 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:566253c9:::13:head -> 13 2024-03-21T06:26:01.928 INFO:tasks.workunit.client.0.smithi096.stdout:: oid=7 cursor=76:5c6b0b28:::7:head 2024-03-21T06:26:01.928 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:5c6b0b28:::7:head 2024-03-21T06:26:01.960 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:5c6b0b28:::7:head -> 7 2024-03-21T06:26:01.961 INFO:tasks.workunit.client.0.smithi096.stdout:: oid=14 cursor=76:62a1935d:::14:head 2024-03-21T06:26:01.961 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:62a1935d:::14:head 2024-03-21T06:26:01.992 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:62a1935d:::14:head -> 14 2024-03-21T06:26:01.992 INFO:tasks.workunit.client.0.smithi096.stdout:: oid=0 cursor=76:6cac518f:::0:head 2024-03-21T06:26:01.992 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:6cac518f:::0:head 2024-03-21T06:26:02.022 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:6cac518f:::0:head -> 0 2024-03-21T06:26:02.022 INFO:tasks.workunit.client.0.smithi096.stdout:: oid=15 cursor=76:863748b0:::15:head 2024-03-21T06:26:02.022 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:863748b0:::15:head 2024-03-21T06:26:02.050 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:863748b0:::15:head -> 15 2024-03-21T06:26:02.050 INFO:tasks.workunit.client.0.smithi096.stdout:: oid=11 cursor=76:89d3ae78:::11:head 2024-03-21T06:26:02.050 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:89d3ae78:::11:head 2024-03-21T06:26:02.073 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:89d3ae78:::11:head -> 11 2024-03-21T06:26:02.073 INFO:tasks.workunit.client.0.smithi096.stdout:: oid=5 cursor=76:b29083e3:::5:head 2024-03-21T06:26:02.074 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:b29083e3:::5:head 2024-03-21T06:26:02.089 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:b29083e3:::5:head -> 5 2024-03-21T06:26:02.089 INFO:tasks.workunit.client.0.smithi096.stdout:: oid=8 cursor=76:bd63b0f1:::8:head 2024-03-21T06:26:02.089 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:bd63b0f1:::8:head 2024-03-21T06:26:02.103 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:bd63b0f1:::8:head -> 8 2024-03-21T06:26:02.103 INFO:tasks.workunit.client.0.smithi096.stdout:: oid=6 cursor=76:c4fdafeb:::6:head 2024-03-21T06:26:02.103 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:c4fdafeb:::6:head 2024-03-21T06:26:02.116 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:c4fdafeb:::6:head -> 6 2024-03-21T06:26:02.116 INFO:tasks.workunit.client.0.smithi096.stdout:: oid=3 cursor=76:cfc208b3:::3:head 2024-03-21T06:26:02.116 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:cfc208b3:::3:head 2024-03-21T06:26:02.126 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:cfc208b3:::3:head -> 3 2024-03-21T06:26:02.126 INFO:tasks.workunit.client.0.smithi096.stdout:: oid=4 cursor=76:d83876eb:::4:head 2024-03-21T06:26:02.127 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:d83876eb:::4:head 2024-03-21T06:26:02.134 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:d83876eb:::4:head -> 4 2024-03-21T06:26:02.135 INFO:tasks.workunit.client.0.smithi096.stdout:: oid=12 cursor=76:de5d7c5f:::12:head 2024-03-21T06:26:02.135 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:de5d7c5f:::12:head 2024-03-21T06:26:02.143 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:de5d7c5f:::12:head -> 12 2024-03-21T06:26:02.143 INFO:tasks.workunit.client.0.smithi096.stdout:: oid=9 cursor=76:e960b815:::9:head 2024-03-21T06:26:02.143 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:e960b815:::9:head 2024-03-21T06:26:02.147 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:e960b815:::9:head -> 9 2024-03-21T06:26:02.148 INFO:tasks.workunit.client.0.smithi096.stdout:: oid=2 cursor=76:f905c69b:::2:head 2024-03-21T06:26:02.148 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:f905c69b:::2:head 2024-03-21T06:26:02.149 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:f905c69b:::2:head -> 2 2024-03-21T06:26:02.149 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:52ea6a34:::10:head 2024-03-21T06:26:02.149 INFO:tasks.workunit.client.0.smithi096.stdout:: cursor()=76:52ea6a34:::10:head expected=76:52ea6a34:::10:head 2024-03-21T06:26:02.188 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:52ea6a34:::10:head -> 10 2024-03-21T06:26:02.188 INFO:tasks.workunit.client.0.smithi096.stdout:: entry=10 expected=10 2024-03-21T06:26:02.188 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:02547ec2:::1:head 2024-03-21T06:26:02.188 INFO:tasks.workunit.client.0.smithi096.stdout:: cursor()=76:02547ec2:::1:head expected=76:02547ec2:::1:head 2024-03-21T06:26:02.239 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:02547ec2:::1:head -> 1 2024-03-21T06:26:02.239 INFO:tasks.workunit.client.0.smithi096.stdout:: entry=1 expected=1 2024-03-21T06:26:02.239 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:f905c69b:::2:head 2024-03-21T06:26:02.239 INFO:tasks.workunit.client.0.smithi096.stdout:: cursor()=76:f905c69b:::2:head expected=76:f905c69b:::2:head 2024-03-21T06:26:02.240 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:f905c69b:::2:head -> 2 2024-03-21T06:26:02.240 INFO:tasks.workunit.client.0.smithi096.stdout:: entry=2 expected=2 2024-03-21T06:26:02.241 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:e960b815:::9:head 2024-03-21T06:26:02.241 INFO:tasks.workunit.client.0.smithi096.stdout:: cursor()=76:e960b815:::9:head expected=76:e960b815:::9:head 2024-03-21T06:26:02.245 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:e960b815:::9:head -> 9 2024-03-21T06:26:02.245 INFO:tasks.workunit.client.0.smithi096.stdout:: entry=9 expected=9 2024-03-21T06:26:02.245 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:de5d7c5f:::12:head 2024-03-21T06:26:02.245 INFO:tasks.workunit.client.0.smithi096.stdout:: cursor()=76:de5d7c5f:::12:head expected=76:de5d7c5f:::12:head 2024-03-21T06:26:02.252 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:de5d7c5f:::12:head -> 12 2024-03-21T06:26:02.253 INFO:tasks.workunit.client.0.smithi096.stdout:: entry=12 expected=12 2024-03-21T06:26:02.253 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:d83876eb:::4:head 2024-03-21T06:26:02.253 INFO:tasks.workunit.client.0.smithi096.stdout:: cursor()=76:d83876eb:::4:head expected=76:d83876eb:::4:head 2024-03-21T06:26:02.260 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:d83876eb:::4:head -> 4 2024-03-21T06:26:02.261 INFO:tasks.workunit.client.0.smithi096.stdout:: entry=4 expected=4 2024-03-21T06:26:02.261 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:cfc208b3:::3:head 2024-03-21T06:26:02.261 INFO:tasks.workunit.client.0.smithi096.stdout:: cursor()=76:cfc208b3:::3:head expected=76:cfc208b3:::3:head 2024-03-21T06:26:02.272 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:cfc208b3:::3:head -> 3 2024-03-21T06:26:02.272 INFO:tasks.workunit.client.0.smithi096.stdout:: entry=3 expected=3 2024-03-21T06:26:02.272 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:c4fdafeb:::6:head 2024-03-21T06:26:02.272 INFO:tasks.workunit.client.0.smithi096.stdout:: cursor()=76:c4fdafeb:::6:head expected=76:c4fdafeb:::6:head 2024-03-21T06:26:02.285 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:c4fdafeb:::6:head -> 6 2024-03-21T06:26:02.285 INFO:tasks.workunit.client.0.smithi096.stdout:: entry=6 expected=6 2024-03-21T06:26:02.286 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:bd63b0f1:::8:head 2024-03-21T06:26:02.286 INFO:tasks.workunit.client.0.smithi096.stdout:: cursor()=76:bd63b0f1:::8:head expected=76:bd63b0f1:::8:head 2024-03-21T06:26:02.299 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:bd63b0f1:::8:head -> 8 2024-03-21T06:26:02.299 INFO:tasks.workunit.client.0.smithi096.stdout:: entry=8 expected=8 2024-03-21T06:26:02.299 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:b29083e3:::5:head 2024-03-21T06:26:02.299 INFO:tasks.workunit.client.0.smithi096.stdout:: cursor()=76:b29083e3:::5:head expected=76:b29083e3:::5:head 2024-03-21T06:26:02.314 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:b29083e3:::5:head -> 5 2024-03-21T06:26:02.314 INFO:tasks.workunit.client.0.smithi096.stdout:: entry=5 expected=5 2024-03-21T06:26:02.314 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:89d3ae78:::11:head 2024-03-21T06:26:02.315 INFO:tasks.workunit.client.0.smithi096.stdout:: cursor()=76:89d3ae78:::11:head expected=76:89d3ae78:::11:head 2024-03-21T06:26:02.357 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:89d3ae78:::11:head -> 11 2024-03-21T06:26:02.357 INFO:tasks.workunit.client.0.smithi096.stdout:: entry=11 expected=11 2024-03-21T06:26:02.357 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:863748b0:::15:head 2024-03-21T06:26:02.357 INFO:tasks.workunit.client.0.smithi096.stdout:: cursor()=76:863748b0:::15:head expected=76:863748b0:::15:head 2024-03-21T06:26:02.381 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:863748b0:::15:head -> 15 2024-03-21T06:26:02.381 INFO:tasks.workunit.client.0.smithi096.stdout:: entry=15 expected=15 2024-03-21T06:26:02.381 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:6cac518f:::0:head 2024-03-21T06:26:02.381 INFO:tasks.workunit.client.0.smithi096.stdout:: cursor()=76:6cac518f:::0:head expected=76:6cac518f:::0:head 2024-03-21T06:26:02.413 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:6cac518f:::0:head -> 0 2024-03-21T06:26:02.413 INFO:tasks.workunit.client.0.smithi096.stdout:: entry=0 expected=0 2024-03-21T06:26:02.413 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:62a1935d:::14:head 2024-03-21T06:26:02.413 INFO:tasks.workunit.client.0.smithi096.stdout:: cursor()=76:62a1935d:::14:head expected=76:62a1935d:::14:head 2024-03-21T06:26:02.444 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:62a1935d:::14:head -> 14 2024-03-21T06:26:02.444 INFO:tasks.workunit.client.0.smithi096.stdout:: entry=14 expected=14 2024-03-21T06:26:02.444 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:5c6b0b28:::7:head 2024-03-21T06:26:02.444 INFO:tasks.workunit.client.0.smithi096.stdout:: cursor()=76:5c6b0b28:::7:head expected=76:5c6b0b28:::7:head 2024-03-21T06:26:02.477 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:5c6b0b28:::7:head -> 7 2024-03-21T06:26:02.477 INFO:tasks.workunit.client.0.smithi096.stdout:: entry=7 expected=7 2024-03-21T06:26:02.477 INFO:tasks.workunit.client.0.smithi096.stdout:: seek to 76:566253c9:::13:head 2024-03-21T06:26:02.477 INFO:tasks.workunit.client.0.smithi096.stdout:: cursor()=76:566253c9:::13:head expected=76:566253c9:::13:head 2024-03-21T06:26:02.511 INFO:tasks.workunit.client.0.smithi096.stdout:> 76:566253c9:::13:head -> 13 2024-03-21T06:26:02.511 INFO:tasks.workunit.client.0.smithi096.stdout:: entry=13 expected=13 2024-03-21T06:26:02.651 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosList.ListObjectsCursor (1089 ms) 2024-03-21T06:26:02.652 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosList.EnumerateObjects 2024-03-21T06:26:02.938 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosList.EnumerateObjects (286 ms) 2024-03-21T06:26:02.938 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosList.EnumerateObjectsSplit 2024-03-21T06:26:03.031 INFO:tasks.workunit.client.0.smithi096.stdout:split 0/5 -> MIN 76:33333333::::head 2024-03-21T06:26:03.044 INFO:tasks.workunit.client.0.smithi096.stdout:split 1/5 -> 76:33333333::::head 76:66666666::::head 2024-03-21T06:26:03.056 INFO:tasks.workunit.client.0.smithi096.stdout:split 2/5 -> 76:66666666::::head 76:99999999::::head 2024-03-21T06:26:03.069 INFO:tasks.workunit.client.0.smithi096.stdout:split 3/5 -> 76:99999999::::head 76:cccccccc::::head 2024-03-21T06:26:03.079 INFO:tasks.workunit.client.0.smithi096.stdout:split 4/5 -> 76:cccccccc::::head MAX 2024-03-21T06:26:03.230 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosList.EnumerateObjectsSplit (293 ms) 2024-03-21T06:26:03.402 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 7 tests from LibRadosList (2838 ms total) 2024-03-21T06:26:03.402 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:26:03.402 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 3 tests from LibRadosListEC 2024-03-21T06:26:03.402 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:70: Skipped 2024-03-21T06:26:03.402 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:03.403 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosListEC.ListObjects 2024-03-21T06:26:03.403 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:84: Skipped 2024-03-21T06:26:03.403 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:03.403 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:96: Skipped 2024-03-21T06:26:03.403 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:03.403 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosListEC.ListObjects (0 ms) 2024-03-21T06:26:03.403 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosListEC.ListObjectsNS 2024-03-21T06:26:03.403 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:84: Skipped 2024-03-21T06:26:03.403 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:03.403 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:96: Skipped 2024-03-21T06:26:03.403 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:03.403 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsNS (0 ms) 2024-03-21T06:26:03.403 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosListEC.ListObjectsStart 2024-03-21T06:26:03.403 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:84: Skipped 2024-03-21T06:26:03.404 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:03.404 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:96: Skipped 2024-03-21T06:26:03.404 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:03.404 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsStart (0 ms) 2024-03-21T06:26:03.404 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:78: Skipped 2024-03-21T06:26:03.404 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:03.404 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 3 tests from LibRadosListEC (0 ms total) 2024-03-21T06:26:03.404 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:26:03.404 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibRadosListNP 2024-03-21T06:26:03.404 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosListNP.ListObjectsError 2024-03-21T06:26:06.415 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosListNP.ListObjectsError (3011 ms) 2024-03-21T06:26:06.415 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibRadosListNP (3012 ms total) 2024-03-21T06:26:06.415 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:26:06.415 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:26:06.415 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 11 tests from 3 test suites ran. (7795 ms total) 2024-03-21T06:26:06.415 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 8 tests. 2024-03-21T06:26:06.415 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] 3 tests, listed below: 2024-03-21T06:26:06.415 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosListEC.ListObjects 2024-03-21T06:26:06.415 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsNS 2024-03-21T06:26:06.415 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsStart 2024-03-21T06:26:06.416 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:26:06.416 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:26:06.416 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_lock 2024-03-21T06:26:06.439 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:26:06.439 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 16 tests from 2 test suites. 2024-03-21T06:26:06.440 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:26:06.440 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 8 tests from LibRadosLock 2024-03-21T06:26:08.420 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLock.LockExclusive 2024-03-21T06:26:08.615 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosLock.LockExclusive (195 ms) 2024-03-21T06:26:08.615 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLock.LockShared 2024-03-21T06:26:08.628 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosLock.LockShared (13 ms) 2024-03-21T06:26:08.628 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLock.LockExclusiveDur 2024-03-21T06:26:09.651 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosLock.LockExclusiveDur (1022 ms) 2024-03-21T06:26:09.651 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLock.LockSharedDur 2024-03-21T06:26:10.676 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosLock.LockSharedDur (1026 ms) 2024-03-21T06:26:10.676 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLock.LockMayRenew 2024-03-21T06:26:10.699 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosLock.LockMayRenew (23 ms) 2024-03-21T06:26:10.699 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLock.Unlock 2024-03-21T06:26:10.725 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosLock.Unlock (26 ms) 2024-03-21T06:26:10.725 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLock.ListLockers 2024-03-21T06:26:10.759 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosLock.ListLockers (33 ms) 2024-03-21T06:26:10.759 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLock.BreakLock 2024-03-21T06:26:10.776 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosLock.BreakLock (18 ms) 2024-03-21T06:26:11.429 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 8 tests from LibRadosLock (2356 ms total) 2024-03-21T06:26:11.429 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:26:11.430 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 8 tests from LibRadosLockEC 2024-03-21T06:26:11.430 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:168: Skipped 2024-03-21T06:26:11.430 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.430 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockEC.LockExclusive 2024-03-21T06:26:11.430 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:26:11.430 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.430 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:26:11.430 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.430 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusive (0 ms) 2024-03-21T06:26:11.430 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockEC.LockShared 2024-03-21T06:26:11.430 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:26:11.430 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.430 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:26:11.431 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.431 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockEC.LockShared (0 ms) 2024-03-21T06:26:11.431 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockEC.LockExclusiveDur 2024-03-21T06:26:11.431 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:26:11.431 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.431 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:26:11.431 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.431 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusiveDur (0 ms) 2024-03-21T06:26:11.431 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockEC.LockSharedDur 2024-03-21T06:26:11.431 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:26:11.431 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.431 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:26:11.432 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.432 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockEC.LockSharedDur (0 ms) 2024-03-21T06:26:11.432 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockEC.LockMayRenew 2024-03-21T06:26:11.432 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:26:11.432 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.432 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:26:11.432 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.432 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockEC.LockMayRenew (0 ms) 2024-03-21T06:26:11.432 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockEC.Unlock 2024-03-21T06:26:11.432 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:26:11.432 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.432 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:26:11.432 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.432 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockEC.Unlock (0 ms) 2024-03-21T06:26:11.433 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockEC.ListLockers 2024-03-21T06:26:11.433 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:26:11.433 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.433 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:26:11.433 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.433 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockEC.ListLockers (0 ms) 2024-03-21T06:26:11.433 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockEC.BreakLock 2024-03-21T06:26:11.433 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:26:11.433 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.433 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:26:11.433 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.434 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockEC.BreakLock (0 ms) 2024-03-21T06:26:11.434 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:176: Skipped 2024-03-21T06:26:11.434 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:11.434 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 8 tests from LibRadosLockEC (0 ms total) 2024-03-21T06:26:11.434 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:26:11.434 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:26:11.434 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 16 tests from 2 test suites ran. (4989 ms total) 2024-03-21T06:26:11.434 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 8 tests. 2024-03-21T06:26:11.434 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] 8 tests, listed below: 2024-03-21T06:26:11.434 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusive 2024-03-21T06:26:11.434 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockEC.LockShared 2024-03-21T06:26:11.434 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusiveDur 2024-03-21T06:26:11.435 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockEC.LockSharedDur 2024-03-21T06:26:11.435 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockEC.LockMayRenew 2024-03-21T06:26:11.435 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockEC.Unlock 2024-03-21T06:26:11.435 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockEC.ListLockers 2024-03-21T06:26:11.435 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockEC.BreakLock 2024-03-21T06:26:11.435 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:26:11.435 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:26:11.435 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_lock_pp 2024-03-21T06:26:11.452 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:26:11.452 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 16 tests from 2 test suites. 2024-03-21T06:26:11.452 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:26:11.452 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 8 tests from LibRadosLockPP 2024-03-21T06:26:11.452 INFO:tasks.workunit.client.0.smithi096.stdout:seed 33991 2024-03-21T06:26:13.441 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockPP.LockExclusivePP 2024-03-21T06:26:13.595 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosLockPP.LockExclusivePP (154 ms) 2024-03-21T06:26:13.596 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockPP.LockSharedPP 2024-03-21T06:26:13.631 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosLockPP.LockSharedPP (35 ms) 2024-03-21T06:26:13.631 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-03-21T06:26:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosLockPP.LockExclusiveDurPP (1036 ms) 2024-03-21T06:26:14.667 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockPP.LockSharedDurPP 2024-03-21T06:26:15.689 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosLockPP.LockSharedDurPP (1022 ms) 2024-03-21T06:26:15.689 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockPP.LockMayRenewPP 2024-03-21T06:26:15.712 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosLockPP.LockMayRenewPP (23 ms) 2024-03-21T06:26:15.712 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockPP.UnlockPP 2024-03-21T06:26:15.738 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosLockPP.UnlockPP (26 ms) 2024-03-21T06:26:15.738 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockPP.ListLockersPP 2024-03-21T06:26:15.769 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosLockPP.ListLockersPP (31 ms) 2024-03-21T06:26:15.770 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockPP.BreakLockPP 2024-03-21T06:26:15.790 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosLockPP.BreakLockPP (21 ms) 2024-03-21T06:26:16.447 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 8 tests from LibRadosLockPP (2349 ms total) 2024-03-21T06:26:16.447 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:26:16.447 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 8 tests from LibRadosLockECPP 2024-03-21T06:26:16.447 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-21T06:26:16.447 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.448 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockECPP.LockExclusivePP 2024-03-21T06:26:16.448 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:26:16.448 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.448 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:26:16.448 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.448 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusivePP (0 ms) 2024-03-21T06:26:16.448 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockECPP.LockSharedPP 2024-03-21T06:26:16.448 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:26:16.448 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.448 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:26:16.448 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.448 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedPP (0 ms) 2024-03-21T06:26:16.448 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-03-21T06:26:16.449 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:26:16.449 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.449 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:26:16.449 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.449 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusiveDurPP (0 ms) 2024-03-21T06:26:16.449 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-03-21T06:26:16.449 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:26:16.449 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.449 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:26:16.449 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.449 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedDurPP (0 ms) 2024-03-21T06:26:16.449 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-03-21T06:26:16.449 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:26:16.450 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.450 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:26:16.450 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.450 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockECPP.LockMayRenewPP (0 ms) 2024-03-21T06:26:16.450 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockECPP.UnlockPP 2024-03-21T06:26:16.450 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:26:16.450 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.450 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:26:16.450 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.450 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockECPP.UnlockPP (0 ms) 2024-03-21T06:26:16.450 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockECPP.ListLockersPP 2024-03-21T06:26:16.450 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:26:16.450 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.450 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:26:16.451 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.451 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockECPP.ListLockersPP (0 ms) 2024-03-21T06:26:16.451 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosLockECPP.BreakLockPP 2024-03-21T06:26:16.451 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:26:16.451 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.451 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:26:16.451 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.451 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockECPP.BreakLockPP (0 ms) 2024-03-21T06:26:16.451 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-21T06:26:16.451 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:26:16.451 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 8 tests from LibRadosLockECPP (0 ms total) 2024-03-21T06:26:16.451 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:26:16.451 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:26:16.451 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 16 tests from 2 test suites ran. (4994 ms total) 2024-03-21T06:26:16.452 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 8 tests. 2024-03-21T06:26:16.452 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] 8 tests, listed below: 2024-03-21T06:26:16.452 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusivePP 2024-03-21T06:26:16.452 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedPP 2024-03-21T06:26:16.452 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusiveDurPP 2024-03-21T06:26:16.452 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedDurPP 2024-03-21T06:26:16.452 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockECPP.LockMayRenewPP 2024-03-21T06:26:16.452 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockECPP.UnlockPP 2024-03-21T06:26:16.452 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockECPP.ListLockersPP 2024-03-21T06:26:16.452 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosLockECPP.BreakLockPP 2024-03-21T06:26:16.453 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:26:16.453 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:26:16.453 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_misc 2024-03-21T06:26:16.480 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:26:16.479+0000 7fc35b231640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:26:16.480 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:26:16.479+0000 7fc35b231640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:26:16.480 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:26:16.479+0000 7fc35b231640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:26:16.480 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 12 tests from 4 test suites. 2024-03-21T06:26:16.480 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:26:16.480 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibRadosMiscVersion 2024-03-21T06:26:16.480 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscVersion.Version 2024-03-21T06:26:16.480 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscVersion.Version (0 ms) 2024-03-21T06:26:16.481 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-03-21T06:26:16.481 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:26:16.481 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from LibRadosMiscConnectFailure 2024-03-21T06:26:16.481 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-03-21T06:26:16.508 INFO:tasks.workunit.client.0.smithi096.stderr:unable to get monitor info from DNS SRV with service name: ceph-mon 2024-03-21T06:26:16.508 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:26:16.508+0000 7fc35b231640 -1 failed for service _ceph-mon._tcp 2024-03-21T06:26:16.508 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:26:16.508+0000 7fc35b231640 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-03-21T06:26:16.508 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscConnectFailure.ConnectFailure (28 ms) 2024-03-21T06:26:16.508 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-03-21T06:26:18.515 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2006 ms) 2024-03-21T06:26:18.515 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from LibRadosMiscConnectFailure (2034 ms total) 2024-03-21T06:26:18.515 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:26:18.515 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibRadosMiscPool 2024-03-21T06:26:18.515 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPool.PoolCreationRace 2024-03-21T06:26:20.632 INFO:tasks.workunit.client.0.smithi096.stdout:started 0x7fc324005d20 2024-03-21T06:26:21.632 INFO:tasks.workunit.client.0.smithi096.stdout:started 0x7fc33c005d20 2024-03-21T06:26:21.632 INFO:tasks.workunit.client.0.smithi096.stdout: started 2 aios 2024-03-21T06:26:21.632 INFO:tasks.workunit.client.0.smithi096.stdout:waiting 0x7fc324005d20 2024-03-21T06:26:21.632 INFO:tasks.workunit.client.0.smithi096.stdout:waiting 0x7fc33c005d20 2024-03-21T06:26:21.634 INFO:tasks.workunit.client.0.smithi096.stdout:done. 2024-03-21T06:26:22.328 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPool.PoolCreationRace (3814 ms) 2024-03-21T06:26:22.328 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibRadosMiscPool (3814 ms total) 2024-03-21T06:26:22.328 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:26:22.328 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 8 tests from LibRadosMisc 2024-03-21T06:26:24.345 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMisc.ClusterFSID 2024-03-21T06:26:24.346 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-03-21T06:26:24.346 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMisc.Exec 2024-03-21T06:26:24.475 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMisc.Exec (130 ms) 2024-03-21T06:26:24.475 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMisc.WriteSame 2024-03-21T06:26:24.553 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMisc.WriteSame (78 ms) 2024-03-21T06:26:24.553 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMisc.CmpExt 2024-03-21T06:26:24.564 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMisc.CmpExt (11 ms) 2024-03-21T06:26:24.564 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMisc.Applications 2024-03-21T06:26:29.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMisc.Applications (4792 ms) 2024-03-21T06:26:29.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMisc.MinCompatOSD 2024-03-21T06:26:29.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-03-21T06:26:29.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMisc.MinCompatClient 2024-03-21T06:26:29.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-03-21T06:26:29.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMisc.ShutdownRace 2024-03-21T06:26:50.614 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMisc.ShutdownRace (21258 ms) 2024-03-21T06:26:51.350 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 8 tests from LibRadosMisc (26269 ms total) 2024-03-21T06:26:51.351 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:26:51.351 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:26:51.351 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 12 tests from 4 test suites ran. (34871 ms total) 2024-03-21T06:26:51.351 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 12 tests. 2024-03-21T06:26:51.363 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:26:51.363 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:26:51.363 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_misc_pp 2024-03-21T06:26:51.392 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:26:51.392+0000 7f5b5a42d7c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:26:51.392 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:26:51.392+0000 7f5b5a42d7c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:26:51.393 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:26:51.392+0000 7f5b5a42d7c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:26:51.393 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 31 tests from 7 test suites. 2024-03-21T06:26:51.393 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:26:51.393 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibRadosMiscVersion 2024-03-21T06:26:51.393 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscVersion.VersionPP 2024-03-21T06:26:51.393 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-03-21T06:26:51.393 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-03-21T06:26:51.393 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:26:51.393 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 22 tests from LibRadosMiscPP 2024-03-21T06:26:51.393 INFO:tasks.workunit.client.0.smithi096.stdout:seed 95657 2024-03-21T06:26:53.332 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-03-21T06:26:53.333 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.WaitOSDMapPP (0 ms) 2024-03-21T06:26:53.333 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.LongNamePP 2024-03-21T06:26:53.587 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.LongNamePP (255 ms) 2024-03-21T06:26:53.587 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.LongLocatorPP 2024-03-21T06:26:53.648 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.LongLocatorPP (61 ms) 2024-03-21T06:26:53.648 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.LongNSpacePP 2024-03-21T06:26:53.675 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.LongNSpacePP (27 ms) 2024-03-21T06:26:53.675 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-03-21T06:26:53.701 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.LongAttrNamePP (26 ms) 2024-03-21T06:26:53.701 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.ExecPP 2024-03-21T06:26:53.709 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.ExecPP (8 ms) 2024-03-21T06:26:53.709 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.BadFlagsPP 2024-03-21T06:26:53.717 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.BadFlagsPP (8 ms) 2024-03-21T06:26:53.717 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.Operate1PP 2024-03-21T06:26:53.732 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.Operate1PP (15 ms) 2024-03-21T06:26:53.732 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.Operate2PP 2024-03-21T06:26:53.743 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.Operate2PP (11 ms) 2024-03-21T06:26:53.743 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.BigObjectPP 2024-03-21T06:26:53.769 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.BigObjectPP (26 ms) 2024-03-21T06:26:53.769 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.AioOperatePP 2024-03-21T06:26:53.779 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.AioOperatePP (10 ms) 2024-03-21T06:26:53.779 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.AssertExistsPP 2024-03-21T06:26:53.793 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.AssertExistsPP (14 ms) 2024-03-21T06:26:53.793 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.AssertVersionPP 2024-03-21T06:26:53.816 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.AssertVersionPP (23 ms) 2024-03-21T06:26:53.816 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.BigAttrPP 2024-03-21T06:26:53.823 INFO:tasks.workunit.client.0.smithi096.stdout:osd_max_attr_size = 0 2024-03-21T06:26:53.823 INFO:tasks.workunit.client.0.smithi096.stdout:osd_max_attr_size == 0; skipping test 2024-03-21T06:27:03.247 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.BigAttrPP (9429 ms) 2024-03-21T06:27:03.247 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.CopyPP 2024-03-21T06:27:03.247 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/misc_cxx.cc:386: Skipped 2024-03-21T06:27:03.247 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:03.247 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosMiscPP.CopyPP (0 ms) 2024-03-21T06:27:03.247 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.CopyScrubPP 2024-03-21T06:27:03.247 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/misc_cxx.cc:549: Skipped 2024-03-21T06:27:03.248 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:03.248 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosMiscPP.CopyScrubPP (0 ms) 2024-03-21T06:27:03.248 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.WriteSamePP 2024-03-21T06:27:03.268 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.WriteSamePP (22 ms) 2024-03-21T06:27:03.268 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.CmpExtPP 2024-03-21T06:27:03.279 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.CmpExtPP (11 ms) 2024-03-21T06:27:03.279 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.Applications 2024-03-21T06:27:07.772 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.Applications (4492 ms) 2024-03-21T06:27:07.772 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.MinCompatOSD 2024-03-21T06:27:07.772 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-03-21T06:27:07.772 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.MinCompatClient 2024-03-21T06:27:07.772 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-03-21T06:27:07.773 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscPP.Conf 2024-03-21T06:27:07.773 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosMiscPP.Conf (0 ms) 2024-03-21T06:27:08.777 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 22 tests from LibRadosMiscPP (14439 ms total) 2024-03-21T06:27:08.777 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:27:08.777 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibRadosTwoPoolsECPP 2024-03-21T06:27:08.777 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/misc_cxx.cc:476: Skipped 2024-03-21T06:27:08.777 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:08.777 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-03-21T06:27:08.778 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/misc_cxx.cc:498: Skipped 2024-03-21T06:27:08.778 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:08.778 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/misc_cxx.cc:504: Skipped 2024-03-21T06:27:08.778 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:08.778 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CopyFrom (0 ms) 2024-03-21T06:27:08.778 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/misc_cxx.cc:491: Skipped 2024-03-21T06:27:08.778 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:08.778 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibRadosTwoPoolsECPP (0 ms total) 2024-03-21T06:27:08.778 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:27:08.778 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-03-21T06:27:10.781 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosChecksum/0.Subset 2024-03-21T06:27:10.953 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosChecksum/0.Subset (172 ms) 2024-03-21T06:27:10.953 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosChecksum/0.Chunked 2024-03-21T06:27:10.973 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosChecksum/0.Chunked (20 ms) 2024-03-21T06:27:11.790 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from LibRadosChecksum/0 (192 ms total) 2024-03-21T06:27:11.790 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:27:11.790 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-03-21T06:27:13.337 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosChecksum/1.Subset 2024-03-21T06:27:13.482 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosChecksum/1.Subset (145 ms) 2024-03-21T06:27:13.482 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosChecksum/1.Chunked 2024-03-21T06:27:13.529 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosChecksum/1.Chunked (47 ms) 2024-03-21T06:27:14.341 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from LibRadosChecksum/1 (192 ms total) 2024-03-21T06:27:14.341 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:27:14.341 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-03-21T06:27:16.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosChecksum/2.Subset 2024-03-21T06:27:16.544 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosChecksum/2.Subset (187 ms) 2024-03-21T06:27:16.544 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosChecksum/2.Chunked 2024-03-21T06:27:16.556 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosChecksum/2.Chunked (12 ms) 2024-03-21T06:27:17.367 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from LibRadosChecksum/2 (199 ms total) 2024-03-21T06:27:17.367 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:27:17.368 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibRadosMiscECPP 2024-03-21T06:27:17.368 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-21T06:27:17.368 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:17.368 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-03-21T06:27:17.368 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:27:17.368 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:17.368 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:27:17.368 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:17.369 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosMiscECPP.CompareExtentRange (0 ms) 2024-03-21T06:27:17.369 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-21T06:27:17.369 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:17.369 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibRadosMiscECPP (0 ms total) 2024-03-21T06:27:17.369 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:27:17.369 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:27:17.369 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 31 tests from 7 test suites ran. (25974 ms total) 2024-03-21T06:27:17.369 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 27 tests. 2024-03-21T06:27:17.370 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] 4 tests, listed below: 2024-03-21T06:27:17.370 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosMiscPP.CopyPP 2024-03-21T06:27:17.370 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosMiscPP.CopyScrubPP 2024-03-21T06:27:17.370 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CopyFrom 2024-03-21T06:27:17.370 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosMiscECPP.CompareExtentRange 2024-03-21T06:27:17.371 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:27:17.371 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:27:17.371 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_tier_pp 2024-03-21T06:27:17.400 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:27:17.400+0000 7f40cc858940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:27:17.401 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:27:17.400+0000 7f40cc858940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:27:17.401 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:27:17.400+0000 7f40cc858940 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:27:17.401 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 77 tests from 4 test suites. 2024-03-21T06:27:17.401 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:27:17.401 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 3 tests from LibRadosTierPP 2024-03-21T06:27:17.401 INFO:tasks.workunit.client.0.smithi096.stdout:seed 95729 2024-03-21T06:27:19.370 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTierPP.Dirty 2024-03-21T06:27:19.371 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:287: Skipped 2024-03-21T06:27:19.371 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:19.371 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTierPP.Dirty (0 ms) 2024-03-21T06:27:19.371 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTierPP.FlushWriteRaces 2024-03-21T06:27:19.371 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:2182: Skipped 2024-03-21T06:27:19.371 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:19.371 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTierPP.FlushWriteRaces (0 ms) 2024-03-21T06:27:19.371 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTierPP.HitSetNone 2024-03-21T06:27:19.371 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:2561: Skipped 2024-03-21T06:27:19.371 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:19.371 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTierPP.HitSetNone (0 ms) 2024-03-21T06:27:20.371 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 3 tests from LibRadosTierPP (0 ms total) 2024-03-21T06:27:20.371 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:27:20.371 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 48 tests from LibRadosTwoPoolsPP 2024-03-21T06:27:22.347 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.Overlay 2024-03-21T06:27:22.347 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.347 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.347 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.348 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.348 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Overlay (0 ms) 2024-03-21T06:27:22.348 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.Promote 2024-03-21T06:27:22.348 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.348 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.348 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.348 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.348 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Promote (0 ms) 2024-03-21T06:27:22.348 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-03-21T06:27:22.348 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.348 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.348 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.348 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.349 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnap (0 ms) 2024-03-21T06:27:22.349 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-03-21T06:27:22.349 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.349 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.349 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.349 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.349 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapScrub (0 ms) 2024-03-21T06:27:22.349 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-03-21T06:27:22.349 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.349 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.349 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.349 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.349 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (0 ms) 2024-03-21T06:27:22.350 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-03-21T06:27:22.350 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.350 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.350 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.350 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.350 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Whiteout (0 ms) 2024-03-21T06:27:22.350 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-03-21T06:27:22.350 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.350 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.350 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.350 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.350 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (0 ms) 2024-03-21T06:27:22.350 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.Evict 2024-03-21T06:27:22.351 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.351 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.351 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.351 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.351 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Evict (0 ms) 2024-03-21T06:27:22.351 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-03-21T06:27:22.351 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.351 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.351 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.351 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.351 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap (0 ms) 2024-03-21T06:27:22.351 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-03-21T06:27:22.351 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.351 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.352 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.352 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.352 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap2 (0 ms) 2024-03-21T06:27:22.352 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-03-21T06:27:22.352 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.352 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.352 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.352 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.352 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ListSnap (0 ms) 2024-03-21T06:27:22.352 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-03-21T06:27:22.352 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.352 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.352 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.352 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.352 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (0 ms) 2024-03-21T06:27:22.353 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-03-21T06:27:22.353 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.353 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.353 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.353 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.353 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlush (0 ms) 2024-03-21T06:27:22.353 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.Flush 2024-03-21T06:27:22.353 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.353 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.353 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.353 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.353 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Flush (0 ms) 2024-03-21T06:27:22.353 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-03-21T06:27:22.353 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.354 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.354 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.354 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.354 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushSnap (0 ms) 2024-03-21T06:27:22.354 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-03-21T06:27:22.354 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.354 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.354 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.354 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.354 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushTryFlushRaces (0 ms) 2024-03-21T06:27:22.354 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-03-21T06:27:22.354 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.354 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.355 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.355 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.355 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlushReadRace (0 ms) 2024-03-21T06:27:22.355 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-03-21T06:27:22.355 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.355 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.355 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.355 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.355 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetRead (0 ms) 2024-03-21T06:27:22.355 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-03-21T06:27:22.355 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.355 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.355 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.355 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.356 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetWrite (0 ms) 2024-03-21T06:27:22.356 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-03-21T06:27:22.356 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.356 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.356 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.356 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.356 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetTrim (0 ms) 2024-03-21T06:27:22.356 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-03-21T06:27:22.356 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.356 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.356 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.356 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.356 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteOn2ndRead (0 ms) 2024-03-21T06:27:22.356 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-03-21T06:27:22.357 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.357 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.357 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.357 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ProxyRead (0 ms) 2024-03-21T06:27:22.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.CachePin 2024-03-21T06:27:22.357 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.357 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.357 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.357 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.CachePin (0 ms) 2024-03-21T06:27:22.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-03-21T06:27:22.357 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.357 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.358 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.358 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.358 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.SetRedirectRead (0 ms) 2024-03-21T06:27:22.358 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-03-21T06:27:22.358 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.358 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.358 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.358 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.358 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestPromoteRead (0 ms) 2024-03-21T06:27:22.358 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-03-21T06:27:22.358 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.358 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.358 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.359 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRefRead (0 ms) 2024-03-21T06:27:22.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-03-21T06:27:22.359 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.359 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.359 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.359 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestUnset (0 ms) 2024-03-21T06:27:22.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-03-21T06:27:22.359 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.359 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.359 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.359 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestDedupRefRead (0 ms) 2024-03-21T06:27:22.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-03-21T06:27:22.360 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.360 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.360 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.360 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount (0 ms) 2024-03-21T06:27:22.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-03-21T06:27:22.360 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.360 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.360 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.360 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (0 ms) 2024-03-21T06:27:22.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-03-21T06:27:22.360 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.361 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.361 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.361 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (0 ms) 2024-03-21T06:27:22.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-03-21T06:27:22.361 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.361 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.361 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.361 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (0 ms) 2024-03-21T06:27:22.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-03-21T06:27:22.361 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.361 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.361 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.362 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (0 ms) 2024-03-21T06:27:22.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-03-21T06:27:22.362 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.362 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.362 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.362 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapIncCount (1 ms) 2024-03-21T06:27:22.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-03-21T06:27:22.362 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.362 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.362 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.362 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvict (0 ms) 2024-03-21T06:27:22.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-03-21T06:27:22.363 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.363 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.363 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.363 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictPromote (0 ms) 2024-03-21T06:27:22.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-03-21T06:27:22.363 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.363 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.363 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.363 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (0 ms) 2024-03-21T06:27:22.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-03-21T06:27:22.363 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.363 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.364 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.364 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.364 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.DedupFlushRead (0 ms) 2024-03-21T06:27:22.364 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-03-21T06:27:22.364 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.364 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.364 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.364 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.364 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushSnap (0 ms) 2024-03-21T06:27:22.364 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-03-21T06:27:22.364 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.364 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.364 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.364 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.365 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushDupCount (0 ms) 2024-03-21T06:27:22.365 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-03-21T06:27:22.365 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.365 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.365 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.365 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.365 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringFlush (0 ms) 2024-03-21T06:27:22.365 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-03-21T06:27:22.365 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.365 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.365 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.365 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.365 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (0 ms) 2024-03-21T06:27:22.365 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-03-21T06:27:22.366 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.366 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.366 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.366 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.366 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollback (0 ms) 2024-03-21T06:27:22.366 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-03-21T06:27:22.366 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.366 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.366 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.366 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.366 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (0 ms) 2024-03-21T06:27:22.366 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-03-21T06:27:22.366 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.366 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.366 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.367 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.367 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictRollback (0 ms) 2024-03-21T06:27:22.367 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-03-21T06:27:22.367 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.367 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.367 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.367 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.367 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PropagateBaseTierError (0 ms) 2024-03-21T06:27:22.367 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-03-21T06:27:22.367 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.367 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.367 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.367 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.367 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HelloWriteReturn (0 ms) 2024-03-21T06:27:22.368 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-03-21T06:27:22.368 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:223: Skipped 2024-03-21T06:27:22.368 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.368 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:233: Skipped 2024-03-21T06:27:22.368 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:22.368 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (0 ms) 2024-03-21T06:27:23.346 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 48 tests from LibRadosTwoPoolsPP (1 ms total) 2024-03-21T06:27:23.346 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:27:23.346 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosTierECPP 2024-03-21T06:27:23.346 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-21T06:27:23.346 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.346 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTierECPP.Dirty 2024-03-21T06:27:23.346 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:27:23.346 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.347 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:27:23.347 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.347 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTierECPP.Dirty (0 ms) 2024-03-21T06:27:23.347 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-03-21T06:27:23.347 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:27:23.347 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.347 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:27:23.347 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.347 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTierECPP.FlushWriteRaces (0 ms) 2024-03-21T06:27:23.347 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTierECPP.CallForcesPromote 2024-03-21T06:27:23.347 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:27:23.347 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.348 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:27:23.348 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.348 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTierECPP.CallForcesPromote (0 ms) 2024-03-21T06:27:23.348 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTierECPP.HitSetNone 2024-03-21T06:27:23.348 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:27:23.348 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.348 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:27:23.348 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.348 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTierECPP.HitSetNone (0 ms) 2024-03-21T06:27:23.348 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-21T06:27:23.348 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.348 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosTierECPP (0 ms total) 2024-03-21T06:27:23.348 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:27:23.348 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 22 tests from LibRadosTwoPoolsECPP 2024-03-21T06:27:23.349 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6267: Skipped 2024-03-21T06:27:23.349 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.349 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-03-21T06:27:23.349 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.349 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.349 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.349 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.349 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Overlay (0 ms) 2024-03-21T06:27:23.349 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.Promote 2024-03-21T06:27:23.349 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.349 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.349 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.349 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.349 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Promote (0 ms) 2024-03-21T06:27:23.349 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-03-21T06:27:23.350 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.350 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.350 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.350 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.350 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnap (0 ms) 2024-03-21T06:27:23.350 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-03-21T06:27:23.350 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.350 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.350 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.350 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.350 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (0 ms) 2024-03-21T06:27:23.350 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-03-21T06:27:23.350 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.351 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.351 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.351 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.351 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Whiteout (0 ms) 2024-03-21T06:27:23.351 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.Evict 2024-03-21T06:27:23.351 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.351 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.351 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.351 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.351 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Evict (0 ms) 2024-03-21T06:27:23.351 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-03-21T06:27:23.351 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.351 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.351 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.352 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.352 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.EvictSnap (0 ms) 2024-03-21T06:27:23.352 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-03-21T06:27:23.352 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.352 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.352 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.352 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.352 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlush (0 ms) 2024-03-21T06:27:23.352 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-03-21T06:27:23.352 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.352 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.352 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.352 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.352 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FailedFlush (0 ms) 2024-03-21T06:27:23.352 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.Flush 2024-03-21T06:27:23.353 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.353 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.353 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.353 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.353 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Flush (0 ms) 2024-03-21T06:27:23.353 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-03-21T06:27:23.353 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.353 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.353 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.353 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.353 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushSnap (0 ms) 2024-03-21T06:27:23.353 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-03-21T06:27:23.353 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.353 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.354 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.354 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.354 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (0 ms) 2024-03-21T06:27:23.354 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-03-21T06:27:23.354 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.354 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.354 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.354 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.354 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlushReadRace (0 ms) 2024-03-21T06:27:23.354 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-03-21T06:27:23.354 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.354 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.354 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.354 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.354 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetRead (0 ms) 2024-03-21T06:27:23.355 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-03-21T06:27:23.355 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.355 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.355 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.355 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.355 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetTrim (0 ms) 2024-03-21T06:27:23.355 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-03-21T06:27:23.355 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.355 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.355 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.355 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.355 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (0 ms) 2024-03-21T06:27:23.355 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-03-21T06:27:23.356 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.356 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.356 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.356 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.356 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ProxyRead (0 ms) 2024-03-21T06:27:23.356 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-03-21T06:27:23.356 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.356 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.356 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.356 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.356 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CachePin (0 ms) 2024-03-21T06:27:23.356 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-03-21T06:27:23.356 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.356 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.356 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.357 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetRedirectRead (0 ms) 2024-03-21T06:27:23.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-03-21T06:27:23.357 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.357 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.357 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.357 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetChunkRead (0 ms) 2024-03-21T06:27:23.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-03-21T06:27:23.357 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.357 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.357 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.357 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ManifestPromoteRead (0 ms) 2024-03-21T06:27:23.357 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-03-21T06:27:23.358 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6278: Skipped 2024-03-21T06:27:23.358 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.358 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6288: Skipped 2024-03-21T06:27:23.358 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.358 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TrySetDedupTier (0 ms) 2024-03-21T06:27:23.358 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/tier_cxx.cc:6272: Skipped 2024-03-21T06:27:23.358 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:23.358 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 22 tests from LibRadosTwoPoolsECPP (1 ms total) 2024-03-21T06:27:23.358 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:27:23.358 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:27:23.358 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 77 tests from 4 test suites ran. (5941 ms total) 2024-03-21T06:27:23.358 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 0 tests. 2024-03-21T06:27:23.358 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] 77 tests, listed below: 2024-03-21T06:27:23.358 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTierPP.Dirty 2024-03-21T06:27:23.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTierPP.FlushWriteRaces 2024-03-21T06:27:23.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTierPP.HitSetNone 2024-03-21T06:27:23.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Overlay 2024-03-21T06:27:23.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Promote 2024-03-21T06:27:23.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnap 2024-03-21T06:27:23.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-03-21T06:27:23.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-03-21T06:27:23.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Whiteout 2024-03-21T06:27:23.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-03-21T06:27:23.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Evict 2024-03-21T06:27:23.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap 2024-03-21T06:27:23.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap2 2024-03-21T06:27:23.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ListSnap 2024-03-21T06:27:23.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-03-21T06:27:23.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlush 2024-03-21T06:27:23.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Flush 2024-03-21T06:27:23.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushSnap 2024-03-21T06:27:23.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-03-21T06:27:23.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-03-21T06:27:23.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetRead 2024-03-21T06:27:23.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetWrite 2024-03-21T06:27:23.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetTrim 2024-03-21T06:27:23.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-03-21T06:27:23.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ProxyRead 2024-03-21T06:27:23.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.CachePin 2024-03-21T06:27:23.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.SetRedirectRead 2024-03-21T06:27:23.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-03-21T06:27:23.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRefRead 2024-03-21T06:27:23.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestUnset 2024-03-21T06:27:23.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-03-21T06:27:23.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-03-21T06:27:23.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-03-21T06:27:23.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-03-21T06:27:23.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-03-21T06:27:23.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-03-21T06:27:23.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-03-21T06:27:23.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvict 2024-03-21T06:27:23.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-03-21T06:27:23.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-03-21T06:27:23.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.DedupFlushRead 2024-03-21T06:27:23.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-03-21T06:27:23.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-03-21T06:27:23.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-03-21T06:27:23.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-03-21T06:27:23.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollback 2024-03-21T06:27:23.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-03-21T06:27:23.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-03-21T06:27:23.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-03-21T06:27:23.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-03-21T06:27:23.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-03-21T06:27:23.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTierECPP.Dirty 2024-03-21T06:27:23.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTierECPP.FlushWriteRaces 2024-03-21T06:27:23.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTierECPP.CallForcesPromote 2024-03-21T06:27:23.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTierECPP.HitSetNone 2024-03-21T06:27:23.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Overlay 2024-03-21T06:27:23.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Promote 2024-03-21T06:27:23.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnap 2024-03-21T06:27:23.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-03-21T06:27:23.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Whiteout 2024-03-21T06:27:23.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Evict 2024-03-21T06:27:23.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.EvictSnap 2024-03-21T06:27:23.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlush 2024-03-21T06:27:23.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FailedFlush 2024-03-21T06:27:23.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Flush 2024-03-21T06:27:23.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushSnap 2024-03-21T06:27:23.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-03-21T06:27:23.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-03-21T06:27:23.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetRead 2024-03-21T06:27:23.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetTrim 2024-03-21T06:27:23.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-03-21T06:27:23.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ProxyRead 2024-03-21T06:27:23.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CachePin 2024-03-21T06:27:23.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-03-21T06:27:23.364 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetChunkRead 2024-03-21T06:27:23.364 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-03-21T06:27:23.364 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-03-21T06:27:23.364 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:27:23.364 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:27:23.364 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_pool 2024-03-21T06:27:23.365 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:27:23.365 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 9 tests from 1 test suite. 2024-03-21T06:27:23.365 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:27:23.365 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 9 tests from LibRadosPools 2024-03-21T06:27:23.365 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosPools.PoolList 2024-03-21T06:27:26.387 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosPools.PoolList (3024 ms) 2024-03-21T06:27:26.387 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosPools.PoolLookup 2024-03-21T06:27:29.400 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosPools.PoolLookup (3013 ms) 2024-03-21T06:27:29.400 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosPools.PoolLookup2 2024-03-21T06:27:32.341 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosPools.PoolLookup2 (2941 ms) 2024-03-21T06:27:32.341 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-03-21T06:27:35.352 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosPools.PoolLookupOtherInstance (3011 ms) 2024-03-21T06:27:35.353 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-03-21T06:27:38.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3010 ms) 2024-03-21T06:27:38.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosPools.PoolDelete 2024-03-21T06:27:43.344 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosPools.PoolDelete (4982 ms) 2024-03-21T06:27:43.344 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosPools.PoolCreateDelete 2024-03-21T06:27:48.393 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosPools.PoolCreateDelete (5049 ms) 2024-03-21T06:27:48.393 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-03-21T06:27:53.348 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosPools.PoolCreateWithCrushRule (4954 ms) 2024-03-21T06:27:53.348 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosPools.PoolGetBaseTier 2024-03-21T06:27:53.348 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/pool.cc:139: Skipped 2024-03-21T06:27:53.348 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:27:53.348 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosPools.PoolGetBaseTier (0 ms) 2024-03-21T06:27:53.348 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 9 tests from LibRadosPools (29984 ms total) 2024-03-21T06:27:53.348 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:27:53.348 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:27:53.348 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 9 tests from 1 test suite ran. (29984 ms total) 2024-03-21T06:27:53.349 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 8 tests. 2024-03-21T06:27:53.349 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] 1 test, listed below: 2024-03-21T06:27:53.349 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosPools.PoolGetBaseTier 2024-03-21T06:27:53.350 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:27:53.350 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:27:53.350 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_snapshots 2024-03-21T06:27:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:27:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 13 tests from 4 test suites. 2024-03-21T06:27:53.369 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:27:53.370 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosSnapshots 2024-03-21T06:27:55.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshots.SnapList 2024-03-21T06:27:57.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosSnapshots.SnapList (2004 ms) 2024-03-21T06:27:57.363 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshots.SnapRemove 2024-03-21T06:27:59.377 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosSnapshots.SnapRemove (2014 ms) 2024-03-21T06:27:59.377 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshots.Rollback 2024-03-21T06:28:01.389 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosSnapshots.Rollback (2012 ms) 2024-03-21T06:28:01.389 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshots.SnapGetName 2024-03-21T06:28:03.344 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosSnapshots.SnapGetName (1955 ms) 2024-03-21T06:28:04.347 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosSnapshots (7985 ms total) 2024-03-21T06:28:04.347 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:28:04.347 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-03-21T06:28:06.356 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-03-21T06:28:10.527 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosSnapshotsSelfManaged.Snap (4171 ms) 2024-03-21T06:28:10.527 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-03-21T06:28:14.507 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosSnapshotsSelfManaged.Rollback (3980 ms) 2024-03-21T06:28:14.508 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-03-21T06:28:19.514 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (5007 ms) 2024-03-21T06:28:20.434 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManaged (13158 ms total) 2024-03-21T06:28:20.435 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:28:20.435 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosSnapshotsEC 2024-03-21T06:28:20.435 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:168: Skipped 2024-03-21T06:28:20.435 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:20.435 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsEC.SnapList 2024-03-21T06:28:20.435 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:28:20.435 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:20.435 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:28:20.435 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:20.435 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapList (1 ms) 2024-03-21T06:28:20.435 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-03-21T06:28:20.436 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:28:20.436 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:20.436 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:28:20.436 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:20.436 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapRemove (0 ms) 2024-03-21T06:28:20.436 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsEC.Rollback 2024-03-21T06:28:20.436 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:28:20.436 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:20.436 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:28:20.436 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:20.436 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsEC.Rollback (0 ms) 2024-03-21T06:28:20.436 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-03-21T06:28:20.437 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:28:20.437 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:20.437 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:28:20.437 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:20.437 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapGetName (0 ms) 2024-03-21T06:28:20.437 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:176: Skipped 2024-03-21T06:28:20.437 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:20.437 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosSnapshotsEC (1 ms total) 2024-03-21T06:28:20.437 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:28:20.437 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-03-21T06:28:20.437 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:168: Skipped 2024-03-21T06:28:20.437 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:20.437 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-03-21T06:28:20.437 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:28:20.438 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:20.438 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:28:20.438 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:20.438 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Snap (0 ms) 2024-03-21T06:28:20.438 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-03-21T06:28:20.438 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:28:20.438 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:20.438 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:28:20.438 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:20.438 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Rollback (0 ms) 2024-03-21T06:28:20.438 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:176: Skipped 2024-03-21T06:28:20.438 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:20.438 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from LibRadosSnapshotsSelfManagedEC (0 ms total) 2024-03-21T06:28:20.439 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:28:20.439 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:28:20.439 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 13 tests from 4 test suites ran. (27064 ms total) 2024-03-21T06:28:20.439 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 7 tests. 2024-03-21T06:28:20.439 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] 6 tests, listed below: 2024-03-21T06:28:20.439 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapList 2024-03-21T06:28:20.439 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapRemove 2024-03-21T06:28:20.439 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsEC.Rollback 2024-03-21T06:28:20.439 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapGetName 2024-03-21T06:28:20.439 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Snap 2024-03-21T06:28:20.439 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-03-21T06:28:20.440 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:28:20.440 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:28:20.440 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_snapshots_pp 2024-03-21T06:28:20.455 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:28:20.456 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 21 tests from 5 test suites. 2024-03-21T06:28:20.456 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:28:20.456 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 5 tests from LibRadosSnapshotsPP 2024-03-21T06:28:20.456 INFO:tasks.workunit.client.0.smithi096.stdout:seed 95985 2024-03-21T06:28:22.352 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-03-21T06:28:24.439 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosSnapshotsPP.SnapListPP (2087 ms) 2024-03-21T06:28:24.439 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-03-21T06:28:26.446 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosSnapshotsPP.SnapRemovePP (2007 ms) 2024-03-21T06:28:26.446 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-03-21T06:28:28.456 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosSnapshotsPP.RollbackPP (2010 ms) 2024-03-21T06:28:28.456 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-03-21T06:28:30.461 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2005 ms) 2024-03-21T06:28:30.462 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-03-21T06:28:34.354 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (3893 ms) 2024-03-21T06:28:35.367 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 5 tests from LibRadosSnapshotsPP (12002 ms total) 2024-03-21T06:28:35.367 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:28:35.367 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-03-21T06:28:37.372 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-03-21T06:28:41.473 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4100 ms) 2024-03-21T06:28:41.473 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-03-21T06:28:41.473 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/snapshots_cxx.cc:156: Skipped 2024-03-21T06:28:41.473 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:41.473 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.RollbackPP (0 ms) 2024-03-21T06:28:41.473 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-03-21T06:28:41.473 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/snapshots_cxx.cc:242: Skipped 2024-03-21T06:28:41.473 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:28:41.473 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (0 ms) 2024-03-21T06:28:41.473 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-03-21T06:28:45.401 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (3929 ms) 2024-03-21T06:28:45.401 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-03-21T06:28:47.495 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2093 ms) 2024-03-21T06:28:47.495 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-21T06:28:47.495 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/snapshots_cxx.cc:463: Skipped 2024-03-21T06:28:47.495 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:28:47.495 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (1 ms) 2024-03-21T06:28:47.495 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-03-21T06:28:49.436 INFO:tasks.workunit.client.0.smithi096.stdout:deleting snap 9 in pool LibRadosSnapshotsSelfManagedPP_smithi096-95985-7 2024-03-21T06:28:50.439 INFO:tasks.workunit.client.0.smithi096.stdout:waiting for snaps to purge 2024-03-21T06:29:05.450 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17955 ms) 2024-03-21T06:29:06.361 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 7 tests from LibRadosSnapshotsSelfManagedPP (28078 ms total) 2024-03-21T06:29:06.361 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:29:06.361 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-03-21T06:29:06.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-03-21T06:29:06.366 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (6 ms) 2024-03-21T06:29:06.366 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-03-21T06:29:11.418 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5050 ms) 2024-03-21T06:29:11.418 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5056 ms total) 2024-03-21T06:29:11.419 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:29:11.419 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosSnapshotsECPP 2024-03-21T06:29:11.419 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-21T06:29:11.419 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.419 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-03-21T06:29:11.419 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:29:11.419 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.419 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:29:11.419 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.419 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapListPP (0 ms) 2024-03-21T06:29:11.419 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-03-21T06:29:11.420 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:29:11.420 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.420 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:29:11.420 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.420 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapRemovePP (0 ms) 2024-03-21T06:29:11.420 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-03-21T06:29:11.420 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:29:11.420 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.420 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:29:11.420 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.420 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.RollbackPP (0 ms) 2024-03-21T06:29:11.420 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-03-21T06:29:11.420 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:29:11.420 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.420 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:29:11.421 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.421 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapGetNamePP (0 ms) 2024-03-21T06:29:11.421 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-21T06:29:11.421 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.421 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosSnapshotsECPP (0 ms total) 2024-03-21T06:29:11.421 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:29:11.421 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-03-21T06:29:11.421 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-21T06:29:11.421 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.421 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-03-21T06:29:11.421 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:29:11.421 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.421 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:29:11.421 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.421 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.SnapPP (0 ms) 2024-03-21T06:29:11.422 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-03-21T06:29:11.422 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:29:11.422 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.422 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:29:11.422 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.422 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (0 ms) 2024-03-21T06:29:11.422 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-03-21T06:29:11.422 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:29:11.422 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.422 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:29:11.422 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.422 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (0 ms) 2024-03-21T06:29:11.422 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-21T06:29:11.423 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:11.423 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (0 ms total) 2024-03-21T06:29:11.423 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:29:11.423 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:29:11.423 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 21 tests from 5 test suites ran. (50961 ms total) 2024-03-21T06:29:11.423 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 11 tests. 2024-03-21T06:29:11.423 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] 10 tests, listed below: 2024-03-21T06:29:11.423 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-03-21T06:29:11.423 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-03-21T06:29:11.423 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-21T06:29:11.423 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapListPP 2024-03-21T06:29:11.423 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapRemovePP 2024-03-21T06:29:11.423 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.RollbackPP 2024-03-21T06:29:11.423 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-03-21T06:29:11.424 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-03-21T06:29:11.424 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-03-21T06:29:11.424 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-03-21T06:29:11.424 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:29:11.424 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:29:11.424 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_stat 2024-03-21T06:29:11.439 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:29:11.439 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 9 tests from 2 test suites. 2024-03-21T06:29:11.439 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:29:11.439 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 5 tests from LibRadosStat 2024-03-21T06:29:13.397 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStat.Stat 2024-03-21T06:29:13.594 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosStat.Stat (198 ms) 2024-03-21T06:29:13.595 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStat.Stat2 2024-03-21T06:29:13.609 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosStat.Stat2 (15 ms) 2024-03-21T06:29:13.609 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStat.StatNS 2024-03-21T06:29:13.640 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosStat.StatNS (31 ms) 2024-03-21T06:29:13.640 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStat.ClusterStat 2024-03-21T06:29:13.641 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosStat.ClusterStat (0 ms) 2024-03-21T06:29:13.641 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStat.PoolStat 2024-03-21T06:29:13.647 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosStat.PoolStat (6 ms) 2024-03-21T06:29:14.397 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 5 tests from LibRadosStat (250 ms total) 2024-03-21T06:29:14.397 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:29:14.397 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosStatEC 2024-03-21T06:29:14.398 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:168: Skipped 2024-03-21T06:29:14.398 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:14.398 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStatEC.Stat 2024-03-21T06:29:14.398 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:29:14.398 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:14.398 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:29:14.398 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:14.398 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosStatEC.Stat (0 ms) 2024-03-21T06:29:14.398 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStatEC.StatNS 2024-03-21T06:29:14.398 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:29:14.398 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:14.398 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:29:14.398 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:14.399 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosStatEC.StatNS (0 ms) 2024-03-21T06:29:14.399 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStatEC.ClusterStat 2024-03-21T06:29:14.399 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:29:14.399 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:14.399 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:29:14.399 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:14.399 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosStatEC.ClusterStat (0 ms) 2024-03-21T06:29:14.399 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStatEC.PoolStat 2024-03-21T06:29:14.399 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:29:14.399 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:14.399 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:29:14.399 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:14.399 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosStatEC.PoolStat (0 ms) 2024-03-21T06:29:14.399 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:176: Skipped 2024-03-21T06:29:14.400 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:14.400 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosStatEC (0 ms total) 2024-03-21T06:29:14.400 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:29:14.400 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:29:14.400 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 9 tests from 2 test suites ran. (2958 ms total) 2024-03-21T06:29:14.400 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 5 tests. 2024-03-21T06:29:14.400 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] 4 tests, listed below: 2024-03-21T06:29:14.400 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosStatEC.Stat 2024-03-21T06:29:14.400 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosStatEC.StatNS 2024-03-21T06:29:14.400 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosStatEC.ClusterStat 2024-03-21T06:29:14.400 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosStatEC.PoolStat 2024-03-21T06:29:14.401 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:29:14.401 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:29:14.401 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_stat_pp 2024-03-21T06:29:14.416 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:29:14.416 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 9 tests from 2 test suites. 2024-03-21T06:29:14.416 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:29:14.416 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 5 tests from LibRadosStatPP 2024-03-21T06:29:14.416 INFO:tasks.workunit.client.0.smithi096.stdout:seed 96120 2024-03-21T06:29:16.404 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStatPP.StatPP 2024-03-21T06:29:16.608 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosStatPP.StatPP (203 ms) 2024-03-21T06:29:16.608 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-03-21T06:29:16.621 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosStatPP.Stat2Mtime2PP (13 ms) 2024-03-21T06:29:16.621 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStatPP.ClusterStatPP 2024-03-21T06:29:16.621 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-03-21T06:29:16.621 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStatPP.PoolStatPP 2024-03-21T06:29:16.627 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosStatPP.PoolStatPP (6 ms) 2024-03-21T06:29:16.627 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStatPP.StatPPNS 2024-03-21T06:29:16.657 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosStatPP.StatPPNS (30 ms) 2024-03-21T06:29:17.359 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 5 tests from LibRadosStatPP (252 ms total) 2024-03-21T06:29:17.359 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:29:17.359 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosStatECPP 2024-03-21T06:29:17.359 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-21T06:29:17.359 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:17.359 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStatECPP.StatPP 2024-03-21T06:29:17.359 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:29:17.359 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:17.360 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:29:17.360 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:17.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosStatECPP.StatPP (0 ms) 2024-03-21T06:29:17.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStatECPP.ClusterStatPP 2024-03-21T06:29:17.360 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:29:17.360 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:17.360 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:29:17.360 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:17.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosStatECPP.ClusterStatPP (0 ms) 2024-03-21T06:29:17.360 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStatECPP.PoolStatPP 2024-03-21T06:29:17.360 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:29:17.360 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:17.361 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:29:17.361 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:17.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosStatECPP.PoolStatPP (0 ms) 2024-03-21T06:29:17.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosStatECPP.StatPPNS 2024-03-21T06:29:17.361 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:29:17.361 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:17.361 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:29:17.361 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:17.361 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosStatECPP.StatPPNS (0 ms) 2024-03-21T06:29:17.361 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-21T06:29:17.361 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:17.361 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosStatECPP (0 ms total) 2024-03-21T06:29:17.362 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:29:17.362 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:29:17.362 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 9 tests from 2 test suites ran. (2942 ms total) 2024-03-21T06:29:17.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 5 tests. 2024-03-21T06:29:17.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] 4 tests, listed below: 2024-03-21T06:29:17.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosStatECPP.StatPP 2024-03-21T06:29:17.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosStatECPP.ClusterStatPP 2024-03-21T06:29:17.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosStatECPP.PoolStatPP 2024-03-21T06:29:17.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosStatECPP.StatPPNS 2024-03-21T06:29:17.363 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:29:17.363 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:29:17.363 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_watch_notify 2024-03-21T06:29:17.377 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:29:17.377 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 11 tests from 2 test suites. 2024-03-21T06:29:17.377 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:29:17.377 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 10 tests from LibRadosWatchNotify 2024-03-21T06:29:19.362 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify 2024-03-21T06:29:19.565 INFO:tasks.workunit.client.0.smithi096.stdout:watch_notify_test_cb 2024-03-21T06:29:19.581 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotify.WatchNotify (219 ms) 2024-03-21T06:29:19.581 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotify.Watch2Delete 2024-03-21T06:29:19.598 INFO:tasks.workunit.client.0.smithi096.stdout:waiting up to 300 for disconnect notification ... 2024-03-21T06:29:22.401 INFO:tasks.workunit.client.0.smithi096.stdout:watch_notify2_test_errcb cookie 94756202655232 err -107 2024-03-21T06:29:22.690 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotify.Watch2Delete (3109 ms) 2024-03-21T06:29:22.690 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-03-21T06:29:22.714 INFO:tasks.workunit.client.0.smithi096.stdout:waiting up to 300 for disconnect notification ... 2024-03-21T06:29:27.401 INFO:tasks.workunit.client.0.smithi096.stdout:watch_notify2_test_errcb cookie 94756202655232 err -107 2024-03-21T06:29:27.721 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotify.AioWatchDelete (5031 ms) 2024-03-21T06:29:27.721 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify2 2024-03-21T06:29:27.743 INFO:tasks.workunit.client.0.smithi096.stdout:watch_notify2_test_cb from 15572 notify_id 1687922147328 cookie 94756202660144 2024-03-21T06:29:27.758 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotify.WatchNotify2 (37 ms) 2024-03-21T06:29:27.758 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-03-21T06:29:27.776 INFO:tasks.workunit.client.0.smithi096.stdout:watch_notify2_test_cb from 15572 notify_id 1687922147328 cookie 94756202660144 2024-03-21T06:29:27.788 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotify.AioWatchNotify2 (31 ms) 2024-03-21T06:29:27.788 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotify.AioNotify 2024-03-21T06:29:27.805 INFO:tasks.workunit.client.0.smithi096.stdout:watch_notify2_test_cb from 15572 notify_id 1687922147328 cookie 94756202694336 2024-03-21T06:29:27.819 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotify.AioNotify (30 ms) 2024-03-21T06:29:27.819 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-03-21T06:29:27.841 INFO:tasks.workunit.client.0.smithi096.stdout:watch_notify2_test_cb from 15572 notify_id 1687922147329 cookie 94756202700720 2024-03-21T06:29:27.841 INFO:tasks.workunit.client.0.smithi096.stdout:watch_notify2_test_cb from 15572 notify_id 1687922147329 cookie 94756202702176 2024-03-21T06:29:27.858 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotify.WatchNotify2Multi (39 ms) 2024-03-21T06:29:27.858 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-03-21T06:29:27.874 INFO:tasks.workunit.client.0.smithi096.stdout:watch_notify2_test_cb from 15572 notify_id 1687922147329 cookie 94756202702016 2024-03-21T06:29:30.874 INFO:tasks.workunit.client.0.smithi096.stdout:watch_notify2_test_cb from 15572 notify_id 1687922147330 cookie 94756202702016 2024-03-21T06:29:30.897 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3039 ms) 2024-03-21T06:29:30.897 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-03-21T06:29:30.913 INFO:tasks.workunit.client.0.smithi096.stdout:waiting up to 1024 for osd to time us out ... 2024-03-21T06:29:34.919 INFO:tasks.workunit.client.0.smithi096.stdout:watch_notify2_test_errcb cookie 94756202700624 err -107 2024-03-21T06:29:35.932 INFO:tasks.workunit.client.0.smithi096.stdout:watch_notify2_test_cb from 15572 notify_id 1687922147330 cookie 94756202700624 2024-03-21T06:29:35.941 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotify.Watch3Timeout (5045 ms) 2024-03-21T06:29:35.941 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-03-21T06:29:35.963 INFO:tasks.workunit.client.0.smithi096.stdout:waiting up to 30 for disconnect notification ... 2024-03-21T06:29:37.403 INFO:tasks.workunit.client.0.smithi096.stdout:watch_notify2_test_errcb cookie 94756202700592 err -107 2024-03-21T06:29:37.969 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotify.AioWatchDelete2 (2027 ms) 2024-03-21T06:29:38.792 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 10 tests from LibRadosWatchNotify (18607 ms total) 2024-03-21T06:29:38.792 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:29:38.792 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibRadosWatchNotifyEC 2024-03-21T06:29:38.792 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:168: Skipped 2024-03-21T06:29:38.792 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:38.792 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-03-21T06:29:38.792 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:182: Skipped 2024-03-21T06:29:38.792 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:38.792 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:196: Skipped 2024-03-21T06:29:38.792 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:38.792 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosWatchNotifyEC.WatchNotify (0 ms) 2024-03-21T06:29:38.792 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/TestCase.cc:176: Skipped 2024-03-21T06:29:38.793 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:38.793 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibRadosWatchNotifyEC (0 ms total) 2024-03-21T06:29:38.793 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:29:38.793 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:29:38.793 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 11 tests from 2 test suites ran. (21415 ms total) 2024-03-21T06:29:38.793 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 10 tests. 2024-03-21T06:29:38.793 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] 1 test, listed below: 2024-03-21T06:29:38.793 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosWatchNotifyEC.WatchNotify 2024-03-21T06:29:38.793 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:29:38.794 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:29:38.794 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_watch_notify_pp 2024-03-21T06:29:38.814 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:29:38.814 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 16 tests from 2 test suites. 2024-03-21T06:29:38.815 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:29:38.815 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from LibRadosWatchNotifyECPP 2024-03-21T06:29:38.815 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:373: Skipped 2024-03-21T06:29:38.815 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:38.815 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-03-21T06:29:38.815 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:29:38.815 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:38.815 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:29:38.815 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:38.815 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotify (0 ms) 2024-03-21T06:29:38.815 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-03-21T06:29:38.815 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:387: Skipped 2024-03-21T06:29:38.815 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:38.815 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:400: Skipped 2024-03-21T06:29:38.816 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:38.816 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (1 ms) 2024-03-21T06:29:38.816 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/testcase_cxx.cc:381: Skipped 2024-03-21T06:29:38.816 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:29:38.816 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from LibRadosWatchNotifyECPP (1 ms total) 2024-03-21T06:29:38.816 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:29:38.816 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-03-21T06:29:40.794 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-03-21T06:29:40.980 INFO:tasks.workunit.client.0.smithi096.stdout:notify 2024-03-21T06:29:40.988 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (194 ms) 2024-03-21T06:29:40.988 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-03-21T06:29:41.010 INFO:tasks.workunit.client.0.smithi096.stdout:notify 2024-03-21T06:29:41.017 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (28 ms) 2024-03-21T06:29:41.017 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-03-21T06:29:41.033 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (16 ms) 2024-03-21T06:29:41.033 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-03-21T06:29:41.053 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (20 ms) 2024-03-21T06:29:41.053 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-03-21T06:29:41.072 INFO:tasks.workunit.client.0.smithi096.stdout:handle_notify cookie 94502712673808 notify_id 1700807049220 notifier_gid 15575 2024-03-21T06:29:41.080 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (27 ms) 2024-03-21T06:29:41.080 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-03-21T06:29:41.095 INFO:tasks.workunit.client.0.smithi096.stdout:handle_notify cookie 94502712673808 notify_id 1700807049217 notifier_gid 15575 2024-03-21T06:29:41.103 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (23 ms) 2024-03-21T06:29:41.103 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-03-21T06:29:41.118 INFO:tasks.workunit.client.0.smithi096.stdout:handle_notify cookie 94502712673808 notify_id 1700807049218 notifier_gid 15575 2024-03-21T06:29:41.126 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (23 ms) 2024-03-21T06:29:41.126 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-03-21T06:29:41.142 INFO:tasks.workunit.client.0.smithi096.stdout:handle_notify cookie 94502712673808 notify_id 1700807049219 notifier_gid 15575 2024-03-21T06:29:41.150 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (24 ms) 2024-03-21T06:29:41.150 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-03-21T06:29:41.166 INFO:tasks.workunit.client.0.smithi096.stdout:handle_notify cookie 94502712673808 notify_id 1700807049220 notifier_gid 15575 2024-03-21T06:29:41.173 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (23 ms) 2024-03-21T06:29:41.173 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-03-21T06:29:41.190 INFO:tasks.workunit.client.0.smithi096.stdout:handle_notify cookie 94502712673808 notify_id 1700807049221 notifier_gid 15575 2024-03-21T06:29:41.197 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (24 ms) 2024-03-21T06:29:41.197 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-03-21T06:29:41.213 INFO:tasks.workunit.client.0.smithi096.stdout: trying... 2024-03-21T06:29:41.214 INFO:tasks.workunit.client.0.smithi096.stdout:handle_notify cookie 94502712681024 notify_id 1700807049218 notifier_gid 15575 2024-03-21T06:29:42.214 INFO:tasks.workunit.client.0.smithi096.stdout: timed out 2024-03-21T06:29:42.221 INFO:tasks.workunit.client.0.smithi096.stdout: flushing 2024-03-21T06:29:44.215 INFO:tasks.workunit.client.0.smithi096.stdout: flushed 2024-03-21T06:29:44.215 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3018 ms) 2024-03-21T06:29:44.215 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-03-21T06:29:44.235 INFO:tasks.workunit.client.0.smithi096.stdout: trying... 2024-03-21T06:29:44.237 INFO:tasks.workunit.client.0.smithi096.stdout:handle_notify cookie 94502712681024 notify_id 1700807049220 notifier_gid 15575 2024-03-21T06:29:45.236 INFO:tasks.workunit.client.0.smithi096.stdout: timed out 2024-03-21T06:29:45.245 INFO:tasks.workunit.client.0.smithi096.stdout: flushing 2024-03-21T06:29:47.238 INFO:tasks.workunit.client.0.smithi096.stdout: flushed 2024-03-21T06:29:47.238 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3023 ms) 2024-03-21T06:29:47.238 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-03-21T06:29:47.258 INFO:tasks.workunit.client.0.smithi096.stdout:List watches 2024-03-21T06:29:47.258 INFO:tasks.workunit.client.0.smithi096.stdout:notify2 2024-03-21T06:29:47.260 INFO:tasks.workunit.client.0.smithi096.stdout:handle_notify cookie 94502712681024 notify_id 1700807049222 notifier_gid 15575 2024-03-21T06:29:50.264 INFO:tasks.workunit.client.0.smithi096.stdout:notify2 done 2024-03-21T06:29:50.264 INFO:tasks.workunit.client.0.smithi096.stdout:watch_check 2024-03-21T06:29:50.264 INFO:tasks.workunit.client.0.smithi096.stdout:unwatch2 2024-03-21T06:29:50.273 INFO:tasks.workunit.client.0.smithi096.stdout: flushing 2024-03-21T06:29:50.273 INFO:tasks.workunit.client.0.smithi096.stdout:done 2024-03-21T06:29:50.273 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3035 ms) 2024-03-21T06:29:50.273 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-03-21T06:29:50.293 INFO:tasks.workunit.client.0.smithi096.stdout:List watches 2024-03-21T06:29:50.293 INFO:tasks.workunit.client.0.smithi096.stdout:notify2 2024-03-21T06:29:50.295 INFO:tasks.workunit.client.0.smithi096.stdout:handle_notify cookie 94502712681024 notify_id 1705102016517 notifier_gid 15575 2024-03-21T06:29:53.299 INFO:tasks.workunit.client.0.smithi096.stdout:notify2 done 2024-03-21T06:29:53.299 INFO:tasks.workunit.client.0.smithi096.stdout:watch_check 2024-03-21T06:29:53.299 INFO:tasks.workunit.client.0.smithi096.stdout:unwatch2 2024-03-21T06:29:53.308 INFO:tasks.workunit.client.0.smithi096.stdout: flushing 2024-03-21T06:29:53.308 INFO:tasks.workunit.client.0.smithi096.stdout:done 2024-03-21T06:29:53.308 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3035 ms) 2024-03-21T06:29:53.375 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (12514 ms total) 2024-03-21T06:29:53.375 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:29:53.375 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:29:53.375 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 16 tests from 2 test suites ran. (14561 ms total) 2024-03-21T06:29:53.375 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 14 tests. 2024-03-21T06:29:53.375 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] 2 tests, listed below: 2024-03-21T06:29:53.376 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotify 2024-03-21T06:29:53.376 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-03-21T06:29:53.377 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:29:53.377 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:29:53.377 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_cmd 2024-03-21T06:29:53.398 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:29:53.398 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 4 tests from 1 test suite. 2024-03-21T06:29:53.398 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:29:53.398 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosCmd 2024-03-21T06:29:53.398 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosCmd.MonDescribe 2024-03-21T06:29:53.428 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosCmd.MonDescribe (31 ms) 2024-03-21T06:29:53.428 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosCmd.OSDCmd 2024-03-21T06:29:53.453 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosCmd.OSDCmd (24 ms) 2024-03-21T06:29:53.453 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosCmd.PGCmd 2024-03-21T06:29:56.390 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosCmd.PGCmd (2936 ms) 2024-03-21T06:29:56.390 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosCmd.WatchLog 2024-03-21T06:29:56.419 INFO:tasks.workunit.client.0.smithi096.stdout:got: 2024-03-21T06:29:55.381935+0000 mon.a [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T06:29:57.393 INFO:tasks.workunit.client.0.smithi096.stdout:got: 2024-03-21T06:29:56.421356+0000 client.admin [INF] onexx 2024-03-21T06:29:57.394 INFO:tasks.workunit.client.0.smithi096.stdout:got: 2024-03-21T06:29:56.421499+0000 mon.a [INF] from='client.? 172.21.15.96:0/3300071783' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-21T06:30:01.397 INFO:tasks.workunit.client.0.smithi096.stdout:got: 2024-03-21T06:29:59.398100+0000 mon.a [INF] from='client.? 172.21.15.96:0/3300071783' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-21T06:30:02.403 INFO:tasks.workunit.client.0.smithi096.stdout:got: 2024-03-21T06:30:01.398629+0000 client.admin [INF] threexx 2024-03-21T06:30:02.404 INFO:tasks.workunit.client.0.smithi096.stdout:got: 2024-03-21T06:30:01.399133+0000 mon.a [INF] from='client.? 172.21.15.96:0/3300071783' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-21T06:30:02.404 INFO:tasks.workunit.client.0.smithi096.stdout:got: 2024-03-21T06:30:02.361205+0000 mon.a [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T06:30:06.409 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosCmd.WatchLog (10019 ms) 2024-03-21T06:30:06.409 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosCmd (13011 ms total) 2024-03-21T06:30:06.409 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:30:06.410 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:30:06.410 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 4 tests from 1 test suite ran. (13011 ms total) 2024-03-21T06:30:06.410 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 4 tests. 2024-03-21T06:30:06.411 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:30:06.411 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:30:06.411 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_cmd_pp 2024-03-21T06:30:06.433 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:30:06.433 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 3 tests from 1 test suite. 2024-03-21T06:30:06.433 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:30:06.433 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 3 tests from LibRadosCmd 2024-03-21T06:30:06.433 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosCmd.MonDescribePP 2024-03-21T06:30:06.460 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosCmd.MonDescribePP (27 ms) 2024-03-21T06:30:06.460 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosCmd.OSDCmdPP 2024-03-21T06:30:06.484 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosCmd.OSDCmdPP (25 ms) 2024-03-21T06:30:06.485 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosCmd.PGCmdPP 2024-03-21T06:30:08.366 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosCmd.PGCmdPP (1881 ms) 2024-03-21T06:30:08.366 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 3 tests from LibRadosCmd (1933 ms total) 2024-03-21T06:30:08.366 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:30:08.366 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:30:08.366 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 3 tests from 1 test suite ran. (1933 ms total) 2024-03-21T06:30:08.366 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 3 tests. 2024-03-21T06:30:08.368 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:30:08.368 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:30:08.368 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_service 2024-03-21T06:30:08.395 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:30:08.395+0000 7f38ba4d47c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:30:08.395 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:30:08.395+0000 7f38ba4d47c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:30:08.395 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:30:08.395+0000 7f38ba4d47c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:30:08.395 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 4 tests from 1 test suite. 2024-03-21T06:30:08.395 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:30:08.395 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosService 2024-03-21T06:30:08.396 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosService.RegisterEarly 2024-03-21T06:30:13.428 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosService.RegisterEarly (5033 ms) 2024-03-21T06:30:13.428 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosService.RegisterLate 2024-03-21T06:30:13.463 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosService.RegisterLate (35 ms) 2024-03-21T06:30:13.463 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosService.StatusFormat 2024-03-21T06:30:13.587 INFO:tasks.workunit.client.0.smithi096.stdout: cluster: 2024-03-21T06:30:13.588 INFO:tasks.workunit.client.0.smithi096.stdout: id: e880df84-2d7c-4386-b5e9-f3879b9ab9d5 2024-03-21T06:30:13.588 INFO:tasks.workunit.client.0.smithi096.stdout: health: HEALTH_WARN 2024-03-21T06:30:13.588 INFO:tasks.workunit.client.0.smithi096.stdout: 1 pool(s) do not have an application enabled 2024-03-21T06:30:13.588 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:30:13.588 INFO:tasks.workunit.client.0.smithi096.stdout: services: 2024-03-21T06:30:13.588 INFO:tasks.workunit.client.0.smithi096.stdout: mon: 1 daemons, quorum a (age 10m) 2024-03-21T06:30:13.588 INFO:tasks.workunit.client.0.smithi096.stdout: mgr: x(active, since 9m) 2024-03-21T06:30:13.588 INFO:tasks.workunit.client.0.smithi096.stdout: osd: 4 osds: 4 up (since 9m), 4 in (since 9m) 2024-03-21T06:30:13.588 INFO:tasks.workunit.client.0.smithi096.stdout: laundry: 1 daemon active (1 hosts) 2024-03-21T06:30:13.588 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:30:13.588 INFO:tasks.workunit.client.0.smithi096.stdout: data: 2024-03-21T06:30:13.588 INFO:tasks.workunit.client.0.smithi096.stdout: pools: 3 pools, 41 pgs 2024-03-21T06:30:13.588 INFO:tasks.workunit.client.0.smithi096.stdout: objects: 3 objects, 577 KiB 2024-03-21T06:30:13.589 INFO:tasks.workunit.client.0.smithi096.stdout: usage: 195 MiB used, 400 GiB / 400 GiB avail 2024-03-21T06:30:13.589 INFO:tasks.workunit.client.0.smithi096.stdout: pgs: 41 active+clean 2024-03-21T06:30:13.589 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:30:13.589 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:30:15.611 INFO:tasks.workunit.client.0.smithi096.stdout: cluster: 2024-03-21T06:30:15.611 INFO:tasks.workunit.client.0.smithi096.stdout: id: e880df84-2d7c-4386-b5e9-f3879b9ab9d5 2024-03-21T06:30:15.611 INFO:tasks.workunit.client.0.smithi096.stdout: health: HEALTH_WARN 2024-03-21T06:30:15.611 INFO:tasks.workunit.client.0.smithi096.stdout: 1 pool(s) do not have an application enabled 2024-03-21T06:30:15.611 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:30:15.611 INFO:tasks.workunit.client.0.smithi096.stdout: services: 2024-03-21T06:30:15.611 INFO:tasks.workunit.client.0.smithi096.stdout: mon: 1 daemons, quorum a (age 10m) 2024-03-21T06:30:15.611 INFO:tasks.workunit.client.0.smithi096.stdout: mgr: x(active, since 9m) 2024-03-21T06:30:15.611 INFO:tasks.workunit.client.0.smithi096.stdout: osd: 4 osds: 4 up (since 9m), 4 in (since 10m) 2024-03-21T06:30:15.611 INFO:tasks.workunit.client.0.smithi096.stdout: foo: 16 portals active (1 hosts, 3 zones) 2024-03-21T06:30:15.611 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:30:15.612 INFO:tasks.workunit.client.0.smithi096.stdout: data: 2024-03-21T06:30:15.612 INFO:tasks.workunit.client.0.smithi096.stdout: pools: 3 pools, 41 pgs 2024-03-21T06:30:15.612 INFO:tasks.workunit.client.0.smithi096.stdout: objects: 3 objects, 577 KiB 2024-03-21T06:30:15.612 INFO:tasks.workunit.client.0.smithi096.stdout: usage: 179 MiB used, 400 GiB / 400 GiB avail 2024-03-21T06:30:15.612 INFO:tasks.workunit.client.0.smithi096.stdout: pgs: 41 active+clean 2024-03-21T06:30:15.612 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:30:15.612 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:30:15.634 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosService.StatusFormat (2171 ms) 2024-03-21T06:30:15.634 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosService.Status 2024-03-21T06:30:35.671 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosService.Status (20037 ms) 2024-03-21T06:30:35.671 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosService (27276 ms total) 2024-03-21T06:30:35.671 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:30:35.671 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:30:35.671 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 4 tests from 1 test suite ran. (27276 ms total) 2024-03-21T06:30:35.671 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 4 tests. 2024-03-21T06:30:35.676 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:30:35.676 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:30:35.676 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_service_pp 2024-03-21T06:30:35.706 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:30:35.706+0000 7ff3eaa697c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:30:35.706 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:30:35.706+0000 7ff3eaa697c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:30:35.706 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:30:35.706+0000 7ff3eaa697c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:30:35.706 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 4 tests from 1 test suite. 2024-03-21T06:30:35.706 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:30:35.706 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosServicePP 2024-03-21T06:30:35.706 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosServicePP.RegisterEarly 2024-03-21T06:30:40.740 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosServicePP.RegisterEarly (5034 ms) 2024-03-21T06:30:40.741 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosServicePP.RegisterLate 2024-03-21T06:30:40.789 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosServicePP.RegisterLate (48 ms) 2024-03-21T06:30:40.789 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosServicePP.Status 2024-03-21T06:31:00.825 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosServicePP.Status (20036 ms) 2024-03-21T06:31:00.825 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosServicePP.Close 2024-03-21T06:31:00.826 INFO:tasks.workunit.client.0.smithi096.stdout:attempt 0 of 20 2024-03-21T06:31:05.883 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosServicePP.Close (5058 ms) 2024-03-21T06:31:05.883 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 4 tests from LibRadosServicePP (30177 ms total) 2024-03-21T06:31:05.883 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:31:05.883 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:31:05.883 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 4 tests from 1 test suite ran. (30177 ms total) 2024-03-21T06:31:05.883 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 4 tests. 2024-03-21T06:31:05.885 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:31:05.885 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:31:05.885 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_c_write_operations 2024-03-21T06:31:05.905 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:31:05.905 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 8 tests from 2 test suites. 2024-03-21T06:31:05.905 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:31:05.905 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibradosCWriteOps 2024-03-21T06:31:05.905 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibradosCWriteOps.NewDelete 2024-03-21T06:31:05.905 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-03-21T06:31:05.906 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from LibradosCWriteOps (0 ms total) 2024-03-21T06:31:05.906 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:31:05.906 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 7 tests from LibRadosCWriteOps 2024-03-21T06:31:05.906 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosCWriteOps.assertExists 2024-03-21T06:31:08.392 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosCWriteOps.assertExists (2486 ms) 2024-03-21T06:31:08.392 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-03-21T06:31:11.407 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosCWriteOps.WriteOpAssertVersion (3015 ms) 2024-03-21T06:31:11.407 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosCWriteOps.Xattrs 2024-03-21T06:31:14.418 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosCWriteOps.Xattrs (3012 ms) 2024-03-21T06:31:14.419 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosCWriteOps.Write 2024-03-21T06:31:17.437 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosCWriteOps.Write (3018 ms) 2024-03-21T06:31:17.437 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosCWriteOps.Exec 2024-03-21T06:31:20.454 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosCWriteOps.Exec (3017 ms) 2024-03-21T06:31:20.454 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosCWriteOps.WriteSame 2024-03-21T06:31:23.465 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosCWriteOps.WriteSame (3011 ms) 2024-03-21T06:31:23.465 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] LibRadosCWriteOps.CmpExt 2024-03-21T06:31:29.811 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] LibRadosCWriteOps.CmpExt (6346 ms) 2024-03-21T06:31:29.811 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 7 tests from LibRadosCWriteOps (23905 ms total) 2024-03-21T06:31:29.811 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:31:29.811 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:31:29.811 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 8 tests from 2 test suites ran. (23905 ms total) 2024-03-21T06:31:29.812 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 8 tests. 2024-03-21T06:31:29.813 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:31:29.813 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:31:29.813 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_c_read_operations 2024-03-21T06:31:29.833 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:31:29.833 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 17 tests from 1 test suite. 2024-03-21T06:31:29.833 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:31:29.833 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 17 tests from CReadOpsTest 2024-03-21T06:31:31.820 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] CReadOpsTest.NewDelete 2024-03-21T06:31:31.820 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] CReadOpsTest.NewDelete (0 ms) 2024-03-21T06:31:31.820 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] CReadOpsTest.SetOpFlags 2024-03-21T06:31:32.034 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] CReadOpsTest.SetOpFlags (215 ms) 2024-03-21T06:31:32.034 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] CReadOpsTest.AssertExists 2024-03-21T06:31:32.057 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] CReadOpsTest.AssertExists (23 ms) 2024-03-21T06:31:32.057 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] CReadOpsTest.AssertVersion 2024-03-21T06:31:32.080 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] CReadOpsTest.AssertVersion (23 ms) 2024-03-21T06:31:32.080 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] CReadOpsTest.CmpXattr 2024-03-21T06:31:32.170 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] CReadOpsTest.CmpXattr (89 ms) 2024-03-21T06:31:32.170 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] CReadOpsTest.Read 2024-03-21T06:31:32.193 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] CReadOpsTest.Read (23 ms) 2024-03-21T06:31:32.193 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] CReadOpsTest.Checksum 2024-03-21T06:31:32.214 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] CReadOpsTest.Checksum (21 ms) 2024-03-21T06:31:32.214 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] CReadOpsTest.RWOrderedRead 2024-03-21T06:31:32.228 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] CReadOpsTest.RWOrderedRead (14 ms) 2024-03-21T06:31:32.228 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] CReadOpsTest.ShortRead 2024-03-21T06:31:32.248 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] CReadOpsTest.ShortRead (20 ms) 2024-03-21T06:31:32.248 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] CReadOpsTest.Exec 2024-03-21T06:31:32.263 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] CReadOpsTest.Exec (16 ms) 2024-03-21T06:31:32.263 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] CReadOpsTest.ExecUserBuf 2024-03-21T06:31:32.281 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] CReadOpsTest.ExecUserBuf (18 ms) 2024-03-21T06:31:32.281 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] CReadOpsTest.Stat 2024-03-21T06:31:32.300 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] CReadOpsTest.Stat (19 ms) 2024-03-21T06:31:32.300 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] CReadOpsTest.Stat2 2024-03-21T06:31:32.319 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] CReadOpsTest.Stat2 (18 ms) 2024-03-21T06:31:32.319 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] CReadOpsTest.Omap 2024-03-21T06:31:32.364 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] CReadOpsTest.Omap (45 ms) 2024-03-21T06:31:32.364 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] CReadOpsTest.OmapNuls 2024-03-21T06:31:32.406 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] CReadOpsTest.OmapNuls (42 ms) 2024-03-21T06:31:32.406 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] CReadOpsTest.GetXattrs 2024-03-21T06:31:32.450 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] CReadOpsTest.GetXattrs (45 ms) 2024-03-21T06:31:32.451 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] CReadOpsTest.CmpExt 2024-03-21T06:31:32.467 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] CReadOpsTest.CmpExt (17 ms) 2024-03-21T06:31:33.382 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 17 tests from CReadOpsTest (648 ms total) 2024-03-21T06:31:33.382 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:31:33.382 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:31:33.383 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 17 tests from 1 test suite ran. (3550 ms total) 2024-03-21T06:31:33.383 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 17 tests. 2024-03-21T06:31:33.384 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:31:33.385 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:31:33.385 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_api_cls_remote_reads 2024-03-21T06:31:33.428 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:31:33.428+0000 7fce56267640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:31:33.428 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:31:33.428+0000 7fce56267640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:31:33.428 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:31:33.429+0000 7fce56267640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:31:33.429 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 1 test from 1 test suite. 2024-03-21T06:31:33.429 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:31:33.429 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from ClsTestRemoteReads 2024-03-21T06:31:33.429 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] ClsTestRemoteReads.TestGather 2024-03-21T06:31:33.429 INFO:tasks.workunit.client.0.smithi096.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2165-g63e11116/rpm/el9/BUILD/ceph-19.0.0-2165-g63e11116/src/test/librados/cls_remote_reads.cc:13: Skipped 2024-03-21T06:31:33.429 INFO:tasks.workunit.client.0.smithi096.stdout:Not supported by crimson yet. Skipped 2024-03-21T06:31:33.429 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] ClsTestRemoteReads.TestGather (0 ms) 2024-03-21T06:31:33.429 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from ClsTestRemoteReads (0 ms total) 2024-03-21T06:31:33.430 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:31:33.430 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:31:33.430 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 1 test from 1 test suite ran. (0 ms total) 2024-03-21T06:31:33.430 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 0 tests. 2024-03-21T06:31:33.430 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] 1 test, listed below: 2024-03-21T06:31:33.430 INFO:tasks.workunit.client.0.smithi096.stdout:[ SKIPPED ] ClsTestRemoteReads.TestGather 2024-03-21T06:31:33.431 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:31:33.431 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:31:33.431 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_list_parallel 2024-03-21T06:31:35.869 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[96862]: starting. 2024-03-21T06:31:35.869 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[96862]: creating pool ceph_test_rados_list_parallel.smithi096-96861 2024-03-21T06:31:35.869 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[96862]: created object 0... 2024-03-21T06:31:35.869 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[96862]: created object 25... 2024-03-21T06:31:35.869 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[96862]: created object 49... 2024-03-21T06:31:35.869 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[96862]: finishing. 2024-03-21T06:31:35.869 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[96862]: shutting down. 2024-03-21T06:31:35.930 INFO:tasks.workunit.client.0.smithi096.stdout:process_2_[96863]: starting. 2024-03-21T06:31:35.931 INFO:tasks.workunit.client.0.smithi096.stdout:process_2_[96863]: listing objects. 2024-03-21T06:31:35.931 INFO:tasks.workunit.client.0.smithi096.stdout:process_2_[96863]: listed object 0... 2024-03-21T06:31:35.931 INFO:tasks.workunit.client.0.smithi096.stdout:process_2_[96863]: listed object 25... 2024-03-21T06:31:35.931 INFO:tasks.workunit.client.0.smithi096.stdout:process_2_[96863]: saw 50 objects 2024-03-21T06:31:35.931 INFO:tasks.workunit.client.0.smithi096.stdout:process_2_[96863]: shutting down. 2024-03-21T06:31:36.259 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:36.259 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[96898]: starting. 2024-03-21T06:31:36.259 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[96898]: creating pool ceph_test_rados_list_parallel.smithi096-96861 2024-03-21T06:31:36.259 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[96898]: created object 0... 2024-03-21T06:31:36.259 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[96898]: created object 25... 2024-03-21T06:31:36.260 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[96898]: created object 49... 2024-03-21T06:31:36.260 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[96898]: finishing. 2024-03-21T06:31:36.260 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[96898]: shutting down. 2024-03-21T06:31:36.456 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:36.456 INFO:tasks.workunit.client.0.smithi096.stdout:process_4_[96899]: starting. 2024-03-21T06:31:36.456 INFO:tasks.workunit.client.0.smithi096.stdout:process_4_[96899]: listing objects. 2024-03-21T06:31:36.456 INFO:tasks.workunit.client.0.smithi096.stdout:process_4_[96899]: listed object 0... 2024-03-21T06:31:36.457 INFO:tasks.workunit.client.0.smithi096.stdout:process_4_[96899]: listed object 25... 2024-03-21T06:31:36.457 INFO:tasks.workunit.client.0.smithi096.stdout:process_4_[96899]: saw 45 objects 2024-03-21T06:31:36.457 INFO:tasks.workunit.client.0.smithi096.stdout:process_4_[96899]: shutting down. 2024-03-21T06:31:36.598 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:36.598 INFO:tasks.workunit.client.0.smithi096.stdout:process_5_[96900]: starting. 2024-03-21T06:31:36.598 INFO:tasks.workunit.client.0.smithi096.stdout:process_5_[96900]: removed 25 objects... 2024-03-21T06:31:36.598 INFO:tasks.workunit.client.0.smithi096.stdout:process_5_[96900]: removed half of the objects 2024-03-21T06:31:36.598 INFO:tasks.workunit.client.0.smithi096.stdout:process_5_[96900]: removed 50 objects... 2024-03-21T06:31:36.599 INFO:tasks.workunit.client.0.smithi096.stdout:process_5_[96900]: removed 50 objects 2024-03-21T06:31:36.599 INFO:tasks.workunit.client.0.smithi096.stdout:process_5_[96900]: shutting down. 2024-03-21T06:31:37.080 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:37.080 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:37.080 INFO:tasks.workunit.client.0.smithi096.stdout:process_6_[96949]: starting. 2024-03-21T06:31:37.080 INFO:tasks.workunit.client.0.smithi096.stdout:process_6_[96949]: creating pool ceph_test_rados_list_parallel.smithi096-96861 2024-03-21T06:31:37.080 INFO:tasks.workunit.client.0.smithi096.stdout:process_6_[96949]: created object 0... 2024-03-21T06:31:37.080 INFO:tasks.workunit.client.0.smithi096.stdout:process_6_[96949]: created object 25... 2024-03-21T06:31:37.080 INFO:tasks.workunit.client.0.smithi096.stdout:process_6_[96949]: created object 49... 2024-03-21T06:31:37.080 INFO:tasks.workunit.client.0.smithi096.stdout:process_6_[96949]: finishing. 2024-03-21T06:31:37.080 INFO:tasks.workunit.client.0.smithi096.stdout:process_6_[96949]: shutting down. 2024-03-21T06:31:37.243 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:37.243 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:37.243 INFO:tasks.workunit.client.0.smithi096.stdout:process_7_[96950]: starting. 2024-03-21T06:31:37.243 INFO:tasks.workunit.client.0.smithi096.stdout:process_7_[96950]: listing objects. 2024-03-21T06:31:37.243 INFO:tasks.workunit.client.0.smithi096.stdout:process_7_[96950]: listed object 0... 2024-03-21T06:31:37.243 INFO:tasks.workunit.client.0.smithi096.stdout:process_7_[96950]: listed object 25... 2024-03-21T06:31:37.243 INFO:tasks.workunit.client.0.smithi096.stdout:process_7_[96950]: listed object 50... 2024-03-21T06:31:37.244 INFO:tasks.workunit.client.0.smithi096.stdout:process_7_[96950]: saw 56 objects 2024-03-21T06:31:37.244 INFO:tasks.workunit.client.0.smithi096.stdout:process_7_[96950]: shutting down. 2024-03-21T06:31:37.402 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:37.402 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:37.402 INFO:tasks.workunit.client.0.smithi096.stdout:process_8_[96951]: starting. 2024-03-21T06:31:37.402 INFO:tasks.workunit.client.0.smithi096.stdout:process_8_[96951]: added 25 objects... 2024-03-21T06:31:37.402 INFO:tasks.workunit.client.0.smithi096.stdout:process_8_[96951]: added half of the objects 2024-03-21T06:31:37.403 INFO:tasks.workunit.client.0.smithi096.stdout:process_8_[96951]: added 50 objects... 2024-03-21T06:31:37.403 INFO:tasks.workunit.client.0.smithi096.stdout:process_8_[96951]: added 50 objects 2024-03-21T06:31:37.403 INFO:tasks.workunit.client.0.smithi096.stdout:process_8_[96951]: shutting down. 2024-03-21T06:31:37.735 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:37.735 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:37.736 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:37.736 INFO:tasks.workunit.client.0.smithi096.stdout:process_9_[97000]: starting. 2024-03-21T06:31:37.736 INFO:tasks.workunit.client.0.smithi096.stdout:process_9_[97000]: creating pool ceph_test_rados_list_parallel.smithi096-96861 2024-03-21T06:31:37.736 INFO:tasks.workunit.client.0.smithi096.stdout:process_9_[97000]: created object 0... 2024-03-21T06:31:37.736 INFO:tasks.workunit.client.0.smithi096.stdout:process_9_[97000]: created object 25... 2024-03-21T06:31:37.736 INFO:tasks.workunit.client.0.smithi096.stdout:process_9_[97000]: created object 49... 2024-03-21T06:31:37.736 INFO:tasks.workunit.client.0.smithi096.stdout:process_9_[97000]: finishing. 2024-03-21T06:31:37.736 INFO:tasks.workunit.client.0.smithi096.stdout:process_9_[97000]: shutting down. 2024-03-21T06:31:37.928 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:37.928 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:37.928 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:37.928 INFO:tasks.workunit.client.0.smithi096.stdout:process_10_[97001]: starting. 2024-03-21T06:31:37.928 INFO:tasks.workunit.client.0.smithi096.stdout:process_10_[97001]: listing objects. 2024-03-21T06:31:37.928 INFO:tasks.workunit.client.0.smithi096.stdout:process_10_[97001]: listed object 0... 2024-03-21T06:31:37.928 INFO:tasks.workunit.client.0.smithi096.stdout:process_10_[97001]: listed object 25... 2024-03-21T06:31:37.928 INFO:tasks.workunit.client.0.smithi096.stdout:process_10_[97001]: listed object 50... 2024-03-21T06:31:37.928 INFO:tasks.workunit.client.0.smithi096.stdout:process_10_[97001]: listed object 75... 2024-03-21T06:31:37.928 INFO:tasks.workunit.client.0.smithi096.stdout:process_10_[97001]: saw 99 objects 2024-03-21T06:31:37.928 INFO:tasks.workunit.client.0.smithi096.stdout:process_10_[97001]: shutting down. 2024-03-21T06:31:38.106 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.107 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.107 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.107 INFO:tasks.workunit.client.0.smithi096.stdout:process_13_[97004]: starting. 2024-03-21T06:31:38.107 INFO:tasks.workunit.client.0.smithi096.stdout:process_13_[97004]: removed 25 objects... 2024-03-21T06:31:38.107 INFO:tasks.workunit.client.0.smithi096.stdout:process_13_[97004]: removed half of the objects 2024-03-21T06:31:38.107 INFO:tasks.workunit.client.0.smithi096.stdout:process_13_[97004]: removed 50 objects... 2024-03-21T06:31:38.107 INFO:tasks.workunit.client.0.smithi096.stdout:process_13_[97004]: removed 50 objects 2024-03-21T06:31:38.107 INFO:tasks.workunit.client.0.smithi096.stdout:process_13_[97004]: shutting down. 2024-03-21T06:31:38.131 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.131 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.131 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.131 INFO:tasks.workunit.client.0.smithi096.stdout:process_11_[97002]: starting. 2024-03-21T06:31:38.131 INFO:tasks.workunit.client.0.smithi096.stdout:process_11_[97002]: added 25 objects... 2024-03-21T06:31:38.131 INFO:tasks.workunit.client.0.smithi096.stdout:process_11_[97002]: added half of the objects 2024-03-21T06:31:38.131 INFO:tasks.workunit.client.0.smithi096.stdout:process_11_[97002]: added 50 objects... 2024-03-21T06:31:38.131 INFO:tasks.workunit.client.0.smithi096.stdout:process_11_[97002]: added 50 objects 2024-03-21T06:31:38.132 INFO:tasks.workunit.client.0.smithi096.stdout:process_11_[97002]: shutting down. 2024-03-21T06:31:38.138 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.139 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.139 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.139 INFO:tasks.workunit.client.0.smithi096.stdout:process_12_[97003]: starting. 2024-03-21T06:31:38.139 INFO:tasks.workunit.client.0.smithi096.stdout:process_12_[97003]: added 25 objects... 2024-03-21T06:31:38.139 INFO:tasks.workunit.client.0.smithi096.stdout:process_12_[97003]: added half of the objects 2024-03-21T06:31:38.139 INFO:tasks.workunit.client.0.smithi096.stdout:process_12_[97003]: added 50 objects... 2024-03-21T06:31:38.139 INFO:tasks.workunit.client.0.smithi096.stdout:process_12_[97003]: added 50 objects 2024-03-21T06:31:38.139 INFO:tasks.workunit.client.0.smithi096.stdout:process_12_[97003]: shutting down. 2024-03-21T06:31:38.468 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.468 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.468 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.468 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.468 INFO:tasks.workunit.client.0.smithi096.stdout:process_14_[97085]: starting. 2024-03-21T06:31:38.469 INFO:tasks.workunit.client.0.smithi096.stdout:process_14_[97085]: creating pool ceph_test_rados_list_parallel.smithi096-96861 2024-03-21T06:31:38.469 INFO:tasks.workunit.client.0.smithi096.stdout:process_14_[97085]: created object 0... 2024-03-21T06:31:38.469 INFO:tasks.workunit.client.0.smithi096.stdout:process_14_[97085]: created object 25... 2024-03-21T06:31:38.469 INFO:tasks.workunit.client.0.smithi096.stdout:process_14_[97085]: created object 49... 2024-03-21T06:31:38.469 INFO:tasks.workunit.client.0.smithi096.stdout:process_14_[97085]: finishing. 2024-03-21T06:31:38.469 INFO:tasks.workunit.client.0.smithi096.stdout:process_14_[97085]: shutting down. 2024-03-21T06:31:38.634 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.634 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.634 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.634 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.634 INFO:tasks.workunit.client.0.smithi096.stdout:process_15_[97086]: starting. 2024-03-21T06:31:38.634 INFO:tasks.workunit.client.0.smithi096.stdout:process_15_[97086]: listing objects. 2024-03-21T06:31:38.634 INFO:tasks.workunit.client.0.smithi096.stdout:process_15_[97086]: listed object 0... 2024-03-21T06:31:38.634 INFO:tasks.workunit.client.0.smithi096.stdout:process_15_[97086]: listed object 25... 2024-03-21T06:31:38.634 INFO:tasks.workunit.client.0.smithi096.stdout:process_15_[97086]: listed object 50... 2024-03-21T06:31:38.634 INFO:tasks.workunit.client.0.smithi096.stdout:process_15_[97086]: listed object 75... 2024-03-21T06:31:38.634 INFO:tasks.workunit.client.0.smithi096.stdout:process_15_[97086]: listed object 100... 2024-03-21T06:31:38.634 INFO:tasks.workunit.client.0.smithi096.stdout:process_15_[97086]: listed object 125... 2024-03-21T06:31:38.635 INFO:tasks.workunit.client.0.smithi096.stdout:process_15_[97086]: saw 150 objects 2024-03-21T06:31:38.635 INFO:tasks.workunit.client.0.smithi096.stdout:process_15_[97086]: shutting down. 2024-03-21T06:31:38.783 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.783 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.784 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.784 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:38.784 INFO:tasks.workunit.client.0.smithi096.stdout:process_16_[97087]: starting. 2024-03-21T06:31:38.784 INFO:tasks.workunit.client.0.smithi096.stdout:process_16_[97087]: added 25 objects... 2024-03-21T06:31:38.784 INFO:tasks.workunit.client.0.smithi096.stdout:process_16_[97087]: added half of the objects 2024-03-21T06:31:38.784 INFO:tasks.workunit.client.0.smithi096.stdout:process_16_[97087]: added 50 objects... 2024-03-21T06:31:38.784 INFO:tasks.workunit.client.0.smithi096.stdout:process_16_[97087]: added 50 objects 2024-03-21T06:31:38.784 INFO:tasks.workunit.client.0.smithi096.stdout:process_16_[97087]: shutting down. 2024-03-21T06:31:39.407 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:39.408 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:39.408 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:39.408 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:39.408 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:39.408 INFO:tasks.workunit.client.0.smithi096.stdout:******* SUCCESS ********** 2024-03-21T06:31:39.409 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:31:39.409 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:31:39.409 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_open_pools_parallel 2024-03-21T06:31:41.977 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[97153]: starting. 2024-03-21T06:31:41.978 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[97153]: creating pool ceph_test_rados_open_pools_parallel.smithi096-97152 2024-03-21T06:31:41.978 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[97153]: created object 0... 2024-03-21T06:31:41.978 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[97153]: created object 25... 2024-03-21T06:31:41.978 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[97153]: created object 49... 2024-03-21T06:31:41.978 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[97153]: finishing. 2024-03-21T06:31:41.978 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[97153]: shutting down. 2024-03-21T06:31:42.384 INFO:tasks.workunit.client.0.smithi096.stdout:process_2_[97154]: starting. 2024-03-21T06:31:42.384 INFO:tasks.workunit.client.0.smithi096.stdout:process_2_[97154]: rados_pool_create. 2024-03-21T06:31:42.384 INFO:tasks.workunit.client.0.smithi096.stdout:process_2_[97154]: rados_ioctx_create. 2024-03-21T06:31:42.384 INFO:tasks.workunit.client.0.smithi096.stdout:process_2_[97154]: shutting down. 2024-03-21T06:31:44.935 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:44.935 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[97189]: starting. 2024-03-21T06:31:44.935 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[97189]: creating pool ceph_test_rados_open_pools_parallel.smithi096-97152 2024-03-21T06:31:44.935 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[97189]: created object 0... 2024-03-21T06:31:44.935 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[97189]: created object 25... 2024-03-21T06:31:44.935 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[97189]: created object 49... 2024-03-21T06:31:44.935 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[97189]: finishing. 2024-03-21T06:31:44.935 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[97189]: shutting down. 2024-03-21T06:31:45.442 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:45.442 INFO:tasks.workunit.client.0.smithi096.stdout:process_4_[97190]: starting. 2024-03-21T06:31:45.442 INFO:tasks.workunit.client.0.smithi096.stdout:process_4_[97190]: rados_pool_create. 2024-03-21T06:31:45.442 INFO:tasks.workunit.client.0.smithi096.stdout:process_4_[97190]: rados_ioctx_create. 2024-03-21T06:31:45.443 INFO:tasks.workunit.client.0.smithi096.stdout:process_4_[97190]: shutting down. 2024-03-21T06:31:45.444 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:45.444 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:45.444 INFO:tasks.workunit.client.0.smithi096.stdout:******* SUCCESS ********** 2024-03-21T06:31:45.445 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T06:31:45.445 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:31:45.445 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_rados_delete_pools_parallel 2024-03-21T06:31:47.952 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[97226]: starting. 2024-03-21T06:31:47.952 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[97226]: creating pool ceph_test_rados_delete_pools_parallel.smithi096-97225 2024-03-21T06:31:47.952 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[97226]: created object 0... 2024-03-21T06:31:47.952 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[97226]: created object 25... 2024-03-21T06:31:47.952 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[97226]: created object 49... 2024-03-21T06:31:47.952 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[97226]: finishing. 2024-03-21T06:31:47.952 INFO:tasks.workunit.client.0.smithi096.stdout:process_1_[97226]: shutting down. 2024-03-21T06:31:48.454 INFO:tasks.workunit.client.0.smithi096.stdout:process_2_[97227]: starting. 2024-03-21T06:31:48.454 INFO:tasks.workunit.client.0.smithi096.stdout:process_2_[97227]: deleting pool ceph_test_rados_delete_pools_parallel.smithi096-97225 2024-03-21T06:31:48.454 INFO:tasks.workunit.client.0.smithi096.stdout:process_2_[97227]: shutting down. 2024-03-21T06:31:50.962 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:50.962 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[97262]: starting. 2024-03-21T06:31:50.962 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[97262]: creating pool ceph_test_rados_delete_pools_parallel.smithi096-97225 2024-03-21T06:31:50.962 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[97262]: created object 0... 2024-03-21T06:31:50.963 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[97262]: created object 25... 2024-03-21T06:31:50.963 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[97262]: created object 49... 2024-03-21T06:31:50.963 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[97262]: finishing. 2024-03-21T06:31:50.963 INFO:tasks.workunit.client.0.smithi096.stdout:process_3_[97262]: shutting down. 2024-03-21T06:31:51.025 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:51.025 INFO:tasks.workunit.client.0.smithi096.stdout:process_5_[97264]: starting. 2024-03-21T06:31:51.026 INFO:tasks.workunit.client.0.smithi096.stdout:process_5_[97264]: listing objects. 2024-03-21T06:31:51.026 INFO:tasks.workunit.client.0.smithi096.stdout:process_5_[97264]: listed object 0... 2024-03-21T06:31:51.026 INFO:tasks.workunit.client.0.smithi096.stdout:process_5_[97264]: listed object 25... 2024-03-21T06:31:51.026 INFO:tasks.workunit.client.0.smithi096.stdout:process_5_[97264]: saw 50 objects 2024-03-21T06:31:51.026 INFO:tasks.workunit.client.0.smithi096.stdout:process_5_[97264]: shutting down. 2024-03-21T06:31:51.469 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:51.469 INFO:tasks.workunit.client.0.smithi096.stdout:process_4_[97263]: starting. 2024-03-21T06:31:51.470 INFO:tasks.workunit.client.0.smithi096.stdout:process_4_[97263]: deleting pool ceph_test_rados_delete_pools_parallel.smithi096-97225 2024-03-21T06:31:51.470 INFO:tasks.workunit.client.0.smithi096.stdout:process_4_[97263]: shutting down. 2024-03-21T06:31:51.471 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:51.471 INFO:tasks.workunit.client.0.smithi096.stdout:******************************* 2024-03-21T06:31:51.471 INFO:tasks.workunit.client.0.smithi096.stdout:******* SUCCESS ********** 2024-03-21T06:31:51.471 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T06:31:51.471 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:31:51.471 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T06:31:51.472 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' cls = ec_io ']' 2024-03-21T06:31:51.472 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' cls = ec_list ']' 2024-03-21T06:31:51.472 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_neorados_cls 2024-03-21T06:31:51.490 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:31:51.490 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 2 tests from 1 test suite. 2024-03-21T06:31:51.490 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:31:51.491 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from NeoRadosCls 2024-03-21T06:31:51.491 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosCls.DNE 2024-03-21T06:31:54.471 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosCls.DNE (2980 ms) 2024-03-21T06:31:54.471 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosCls.RemoteReads 2024-03-21T06:31:55.473 INFO:tasks.workunit.client.0.smithi096.stderr:Not supported by crimson yet. Skipped 2024-03-21T06:31:56.475 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosCls.RemoteReads (2005 ms) 2024-03-21T06:31:56.475 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from NeoRadosCls (4985 ms total) 2024-03-21T06:31:56.475 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:31:56.476 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:31:56.476 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 2 tests from 1 test suite ran. (4985 ms total) 2024-03-21T06:31:56.476 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 2 tests. 2024-03-21T06:31:56.477 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T06:31:56.477 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:31:56.477 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T06:31:56.477 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' cmd = ec_io ']' 2024-03-21T06:31:56.477 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' cmd = ec_list ']' 2024-03-21T06:31:56.478 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_neorados_cmd 2024-03-21T06:31:56.499 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:31:56.499 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 3 tests from 1 test suite. 2024-03-21T06:31:56.499 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:31:56.499 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 3 tests from NeoRadosCmd 2024-03-21T06:31:56.499 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosCmd.MonDescribe 2024-03-21T06:31:58.489 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosCmd.MonDescribe (1990 ms) 2024-03-21T06:31:58.489 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosCmd.OSDCmd 2024-03-21T06:32:00.493 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosCmd.OSDCmd (2004 ms) 2024-03-21T06:32:00.493 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosCmd.PGCmd 2024-03-21T06:32:03.506 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosCmd.PGCmd (3013 ms) 2024-03-21T06:32:03.506 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 3 tests from NeoRadosCmd (7007 ms total) 2024-03-21T06:32:03.506 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:32:03.506 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:32:03.506 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 3 tests from 1 test suite ran. (7007 ms total) 2024-03-21T06:32:03.506 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 3 tests. 2024-03-21T06:32:03.508 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T06:32:03.508 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:32:03.508 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T06:32:03.508 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' handler_error = ec_io ']' 2024-03-21T06:32:03.508 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' handler_error = ec_list ']' 2024-03-21T06:32:03.508 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_neorados_handler_error 2024-03-21T06:32:03.529 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:32:03.529 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 1 test from 1 test suite. 2024-03-21T06:32:03.529 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:32:03.529 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from neocls_handler_error 2024-03-21T06:32:03.529 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] neocls_handler_error.test_handler_error 2024-03-21T06:32:06.521 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] neocls_handler_error.test_handler_error (2993 ms) 2024-03-21T06:32:06.521 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 1 test from neocls_handler_error (2993 ms total) 2024-03-21T06:32:06.522 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:32:06.522 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:32:06.522 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 1 test from 1 test suite ran. (2993 ms total) 2024-03-21T06:32:06.522 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 1 test. 2024-03-21T06:32:06.523 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T06:32:06.523 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:32:06.523 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T06:32:06.524 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' io = ec_io ']' 2024-03-21T06:32:06.524 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' io = ec_list ']' 2024-03-21T06:32:06.524 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_neorados_io 2024-03-21T06:32:06.543 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:32:06.543 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 14 tests from 1 test suite. 2024-03-21T06:32:06.543 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:32:06.543 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 14 tests from NeoRadosIo 2024-03-21T06:32:06.543 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosIo.Limits 2024-03-21T06:32:07.521 INFO:tasks.workunit.client.0.smithi096.stderr:Not supported by crimson yet. Skipped 2024-03-21T06:32:08.523 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosIo.Limits (1979 ms) 2024-03-21T06:32:08.523 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosIo.SimpleWrite 2024-03-21T06:32:11.538 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosIo.SimpleWrite (3016 ms) 2024-03-21T06:32:11.539 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosIo.ReadOp 2024-03-21T06:32:14.395 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosIo.ReadOp (2857 ms) 2024-03-21T06:32:14.396 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosIo.SparseRead 2024-03-21T06:32:17.392 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosIo.SparseRead (2996 ms) 2024-03-21T06:32:17.392 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosIo.RoundTrip 2024-03-21T06:32:20.440 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosIo.RoundTrip (3049 ms) 2024-03-21T06:32:20.441 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-03-21T06:32:23.452 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosIo.ReadIntoBuufferlist (3011 ms) 2024-03-21T06:32:23.452 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-03-21T06:32:26.465 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosIo.OverlappingWriteRoundTrip (3014 ms) 2024-03-21T06:32:26.466 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-03-21T06:32:29.474 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosIo.WriteFullRoundTrip (3008 ms) 2024-03-21T06:32:29.474 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosIo.AppendRoundTrip 2024-03-21T06:32:32.485 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosIo.AppendRoundTrip (3011 ms) 2024-03-21T06:32:32.485 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosIo.Trunc 2024-03-21T06:32:35.497 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosIo.Trunc (3012 ms) 2024-03-21T06:32:35.497 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosIo.Remove 2024-03-21T06:32:38.511 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosIo.Remove (3014 ms) 2024-03-21T06:32:38.511 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosIo.XattrsRoundTrip 2024-03-21T06:32:41.534 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosIo.XattrsRoundTrip (3024 ms) 2024-03-21T06:32:41.534 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosIo.RmXattr 2024-03-21T06:32:44.542 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosIo.RmXattr (3007 ms) 2024-03-21T06:32:44.542 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosIo.GetXattrs 2024-03-21T06:32:47.396 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosIo.GetXattrs (2854 ms) 2024-03-21T06:32:47.396 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 14 tests from NeoRadosIo (40852 ms total) 2024-03-21T06:32:47.396 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:32:47.396 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:32:47.396 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 14 tests from 1 test suite ran. (40852 ms total) 2024-03-21T06:32:47.396 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 14 tests. 2024-03-21T06:32:47.399 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T06:32:47.399 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:32:47.399 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T06:32:47.399 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' ec_io = ec_io ']' 2024-03-21T06:32:47.399 INFO:tasks.workunit.client.0.smithi096.stdout:Skipping EC with Crimson 2024-03-21T06:32:47.400 INFO:tasks.workunit.client.0.smithi096.stderr:+ echo 'Skipping EC with Crimson' 2024-03-21T06:32:47.400 INFO:tasks.workunit.client.0.smithi096.stderr:+ continue 2024-03-21T06:32:47.400 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T06:32:47.400 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:32:47.400 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T06:32:47.400 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' list = ec_io ']' 2024-03-21T06:32:47.400 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' list = ec_list ']' 2024-03-21T06:32:47.400 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_neorados_list 2024-03-21T06:32:47.416 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:32:47.416 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 3 tests from 1 test suite. 2024-03-21T06:32:47.416 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:32:47.416 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 3 tests from NeoradosList 2024-03-21T06:32:47.416 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoradosList.ListObjects 2024-03-21T06:32:50.403 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoradosList.ListObjects (2987 ms) 2024-03-21T06:32:50.403 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoradosList.ListObjectsNS 2024-03-21T06:32:53.417 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoradosList.ListObjectsNS (3015 ms) 2024-03-21T06:32:53.418 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoradosList.ListObjectsMany 2024-03-21T06:32:59.829 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoradosList.ListObjectsMany (6411 ms) 2024-03-21T06:32:59.829 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 3 tests from NeoradosList (12413 ms total) 2024-03-21T06:32:59.829 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:32:59.829 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:32:59.829 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 3 tests from 1 test suite ran. (12413 ms total) 2024-03-21T06:32:59.829 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 3 tests. 2024-03-21T06:32:59.831 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T06:32:59.831 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:32:59.831 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T06:32:59.831 INFO:tasks.workunit.client.0.smithi096.stdout:Skipping EC with Crimson 2024-03-21T06:32:59.832 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' ec_list = ec_io ']' 2024-03-21T06:32:59.832 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' ec_list = ec_list ']' 2024-03-21T06:32:59.832 INFO:tasks.workunit.client.0.smithi096.stderr:+ echo 'Skipping EC with Crimson' 2024-03-21T06:32:59.832 INFO:tasks.workunit.client.0.smithi096.stderr:+ continue 2024-03-21T06:32:59.832 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T06:32:59.832 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:32:59.832 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T06:32:59.832 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' misc = ec_io ']' 2024-03-21T06:32:59.832 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' misc = ec_list ']' 2024-03-21T06:32:59.832 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_neorados_misc 2024-03-21T06:32:59.846 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:32:59.846 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 12 tests from 1 test suite. 2024-03-21T06:32:59.846 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:32:59.847 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 12 tests from NeoRadosMisc 2024-03-21T06:32:59.847 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosMisc.Version 2024-03-21T06:33:01.835 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosMisc.Version (1989 ms) 2024-03-21T06:33:01.835 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosMisc.WaitOSDMap 2024-03-21T06:33:03.405 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosMisc.WaitOSDMap (1569 ms) 2024-03-21T06:33:03.405 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosMisc.LongName 2024-03-21T06:33:06.419 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosMisc.LongName (3014 ms) 2024-03-21T06:33:06.419 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosMisc.LongLocator 2024-03-21T06:33:09.443 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosMisc.LongLocator (3024 ms) 2024-03-21T06:33:09.443 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosMisc.LongNamespace 2024-03-21T06:33:12.401 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosMisc.LongNamespace (2958 ms) 2024-03-21T06:33:12.401 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosMisc.LongAttrName 2024-03-21T06:33:15.460 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosMisc.LongAttrName (3059 ms) 2024-03-21T06:33:15.460 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosMisc.Exec 2024-03-21T06:33:18.468 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosMisc.Exec (3009 ms) 2024-03-21T06:33:18.468 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosMisc.Operate1 2024-03-21T06:33:21.486 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosMisc.Operate1 (3017 ms) 2024-03-21T06:33:21.486 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosMisc.Operate2 2024-03-21T06:33:24.493 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosMisc.Operate2 (3008 ms) 2024-03-21T06:33:24.494 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosMisc.BigObject 2024-03-21T06:33:27.512 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosMisc.BigObject (3019 ms) 2024-03-21T06:33:27.513 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosMisc.BigAttr 2024-03-21T06:33:29.521 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosMisc.BigAttr (2009 ms) 2024-03-21T06:33:29.522 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosMisc.WriteSame 2024-03-21T06:33:30.525 INFO:tasks.workunit.client.0.smithi096.stderr:Not supported by crimson yet. Skipped 2024-03-21T06:33:31.529 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosMisc.WriteSame (2007 ms) 2024-03-21T06:33:31.529 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 12 tests from NeoRadosMisc (31682 ms total) 2024-03-21T06:33:31.529 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:33:31.529 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:33:31.529 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 12 tests from 1 test suite ran. (31682 ms total) 2024-03-21T06:33:31.530 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 12 tests. 2024-03-21T06:33:31.533 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T06:33:31.533 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:33:31.533 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T06:33:31.533 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' pool = ec_io ']' 2024-03-21T06:33:31.533 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' pool = ec_list ']' 2024-03-21T06:33:31.533 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_neorados_pool 2024-03-21T06:33:31.557 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:33:31.557 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 6 tests from 1 test suite. 2024-03-21T06:33:31.558 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:33:31.558 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 6 tests from NeoRadosPools 2024-03-21T06:33:31.558 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosPools.PoolList 2024-03-21T06:33:33.411 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosPools.PoolList (1854 ms) 2024-03-21T06:33:33.411 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosPools.PoolLookup 2024-03-21T06:33:35.417 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosPools.PoolLookup (2006 ms) 2024-03-21T06:33:35.418 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-03-21T06:33:37.431 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosPools.PoolLookupOtherInstance (2013 ms) 2024-03-21T06:33:37.431 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosPools.PoolDelete 2024-03-21T06:33:41.448 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosPools.PoolDelete (4017 ms) 2024-03-21T06:33:41.448 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosPools.PoolCreateDelete 2024-03-21T06:33:43.410 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosPools.PoolCreateDelete (1962 ms) 2024-03-21T06:33:43.410 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-03-21T06:33:45.422 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosPools.PoolCreateWithCrushRule (2013 ms) 2024-03-21T06:33:45.423 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 6 tests from NeoRadosPools (13865 ms total) 2024-03-21T06:33:45.423 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:33:45.423 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:33:45.423 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 6 tests from 1 test suite ran. (13865 ms total) 2024-03-21T06:33:45.423 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 6 tests. 2024-03-21T06:33:45.425 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T06:33:45.425 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:33:45.425 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T06:33:45.426 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' read_operations = ec_io ']' 2024-03-21T06:33:45.426 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' read_operations = ec_list ']' 2024-03-21T06:33:45.426 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_neorados_read_operations 2024-03-21T06:33:45.449 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:33:45.449 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 14 tests from 1 test suite. 2024-03-21T06:33:45.449 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:33:45.449 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 14 tests from NeoRadosReadOps 2024-03-21T06:33:45.449 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosReadOps.SetOpFlags 2024-03-21T06:33:48.434 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosReadOps.SetOpFlags (2983 ms) 2024-03-21T06:33:48.434 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosReadOps.AssertExists 2024-03-21T06:33:51.446 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosReadOps.AssertExists (3014 ms) 2024-03-21T06:33:51.447 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosReadOps.AssertVersion 2024-03-21T06:33:54.453 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosReadOps.AssertVersion (3006 ms) 2024-03-21T06:33:54.453 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosReadOps.CmpXattr 2024-03-21T06:33:57.467 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosReadOps.CmpXattr (3015 ms) 2024-03-21T06:33:57.467 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosReadOps.Read 2024-03-21T06:34:00.479 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosReadOps.Read (3011 ms) 2024-03-21T06:34:00.479 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosReadOps.Checksum 2024-03-21T06:34:03.493 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosReadOps.Checksum (3015 ms) 2024-03-21T06:34:03.493 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosReadOps.RWOrderedRead 2024-03-21T06:34:06.503 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosReadOps.RWOrderedRead (3010 ms) 2024-03-21T06:34:06.504 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosReadOps.ShortRead 2024-03-21T06:34:09.519 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosReadOps.ShortRead (3016 ms) 2024-03-21T06:34:09.519 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosReadOps.Exec 2024-03-21T06:34:12.416 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosReadOps.Exec (2896 ms) 2024-03-21T06:34:12.416 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosReadOps.Stat 2024-03-21T06:34:15.533 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosReadOps.Stat (3118 ms) 2024-03-21T06:34:15.533 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosReadOps.Omap 2024-03-21T06:34:18.548 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosReadOps.Omap (3015 ms) 2024-03-21T06:34:18.548 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosReadOps.OmapNuls 2024-03-21T06:34:21.560 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosReadOps.OmapNuls (3012 ms) 2024-03-21T06:34:21.560 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosReadOps.GetXattrs 2024-03-21T06:34:24.562 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosReadOps.GetXattrs (3002 ms) 2024-03-21T06:34:24.563 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosReadOps.CmpExt 2024-03-21T06:34:27.575 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosReadOps.CmpExt (3013 ms) 2024-03-21T06:34:27.575 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 14 tests from NeoRadosReadOps (42126 ms total) 2024-03-21T06:34:27.575 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:34:27.575 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:34:27.575 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 14 tests from 1 test suite ran. (42126 ms total) 2024-03-21T06:34:27.576 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 14 tests. 2024-03-21T06:34:27.579 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T06:34:27.579 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:34:27.579 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T06:34:27.580 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' snapshots = ec_io ']' 2024-03-21T06:34:27.580 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' snapshots = ec_list ']' 2024-03-21T06:34:27.580 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_neorados_snapshots 2024-03-21T06:34:27.599 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:34:27.599 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 11 tests from 2 test suites. 2024-03-21T06:34:27.599 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:34:27.599 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 5 tests from NeoRadosSnapshots 2024-03-21T06:34:27.599 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosSnapshots.SnapList 2024-03-21T06:34:32.419 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosSnapshots.SnapList (4821 ms) 2024-03-21T06:34:32.420 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosSnapshots.SnapRemove 2024-03-21T06:34:37.432 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosSnapshots.SnapRemove (5012 ms) 2024-03-21T06:34:37.432 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosSnapshots.Rollback 2024-03-21T06:34:41.459 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosSnapshots.Rollback (4028 ms) 2024-03-21T06:34:41.459 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosSnapshots.SnapGetName 2024-03-21T06:34:46.478 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosSnapshots.SnapGetName (5019 ms) 2024-03-21T06:34:46.479 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-03-21T06:34:53.428 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosSnapshots.SnapCreateRemove (6950 ms) 2024-03-21T06:34:53.428 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 5 tests from NeoRadosSnapshots (25830 ms total) 2024-03-21T06:34:53.428 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:34:53.428 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 6 tests from NeoRadosSelfManagedSnaps 2024-03-21T06:34:53.429 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-03-21T06:34:58.447 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosSelfManagedSnaps.Snap (5019 ms) 2024-03-21T06:34:58.448 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-03-21T06:34:59.454 INFO:tasks.workunit.client.0.smithi096.stderr:Not supported by crimson yet. Skipped 2024-03-21T06:35:00.455 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosSelfManagedSnaps.Rollback (2008 ms) 2024-03-21T06:35:00.455 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-03-21T06:35:01.462 INFO:tasks.workunit.client.0.smithi096.stderr:Not supported by crimson yet. Skipped 2024-03-21T06:35:02.426 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (1971 ms) 2024-03-21T06:35:02.427 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-03-21T06:35:08.448 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosSelfManagedSnaps.Bug11677 (6022 ms) 2024-03-21T06:35:08.448 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-03-21T06:35:12.428 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosSelfManagedSnaps.OrderSnap (3980 ms) 2024-03-21T06:35:12.429 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-03-21T06:35:15.436 INFO:tasks.workunit.client.0.smithi096.stdout:Deleting snap 3 in pool ReusePurgedSnapsmithi096-98171-11. 2024-03-21T06:35:16.438 INFO:tasks.workunit.client.0.smithi096.stdout:Waiting for snaps to purge. 2024-03-21T06:35:32.440 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (20011 ms) 2024-03-21T06:35:32.440 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 6 tests from NeoRadosSelfManagedSnaps (39011 ms total) 2024-03-21T06:35:32.440 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:35:32.440 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:35:32.440 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 11 tests from 2 test suites ran. (64841 ms total) 2024-03-21T06:35:32.440 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 11 tests. 2024-03-21T06:35:32.443 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T06:35:32.443 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:35:32.443 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T06:35:32.444 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' watch_notify = ec_io ']' 2024-03-21T06:35:32.444 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' watch_notify = ec_list ']' 2024-03-21T06:35:32.444 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_neorados_watch_notify 2024-03-21T06:35:32.462 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:35:32.463 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 2 tests from 1 test suite. 2024-03-21T06:35:32.463 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:35:32.463 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from NeoRadosWatchNotify 2024-03-21T06:35:32.463 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosWatchNotify.WatchNotify 2024-03-21T06:35:34.728 INFO:tasks.workunit.client.0.smithi096.stdout:handle_notify cookie 94248138996560 notify_id 2804613644291 notifier_gid 15971 2024-03-21T06:35:35.453 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosWatchNotify.WatchNotify (2990 ms) 2024-03-21T06:35:35.453 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-03-21T06:35:37.696 INFO:tasks.workunit.client.0.smithi096.stdout:Trying... 2024-03-21T06:35:37.697 INFO:tasks.workunit.client.0.smithi096.stdout:handle_notify cookie 94248151500880 notify_id 2817498546181 notifier_gid 15974 2024-03-21T06:35:37.698 INFO:tasks.workunit.client.0.smithi096.stdout:Waiting for 3.000000000s 2024-03-21T06:35:38.698 INFO:tasks.workunit.client.0.smithi096.stdout:Timed out. 2024-03-21T06:35:38.705 INFO:tasks.workunit.client.0.smithi096.stdout:Flushing... 2024-03-21T06:35:38.705 INFO:tasks.workunit.client.0.smithi096.stdout:Flushed... 2024-03-21T06:35:41.865 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (6413 ms) 2024-03-21T06:35:41.866 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 2 tests from NeoRadosWatchNotify (9403 ms total) 2024-03-21T06:35:41.866 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:35:41.866 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:35:41.866 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 2 tests from 1 test suite ran. (9403 ms total) 2024-03-21T06:35:41.866 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 2 tests. 2024-03-21T06:35:41.868 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T06:35:41.868 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:35:41.868 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T06:35:41.868 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' write_operations = ec_io ']' 2024-03-21T06:35:41.868 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' write_operations = ec_list ']' 2024-03-21T06:35:41.868 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph_test_neorados_write_operations 2024-03-21T06:35:41.883 INFO:tasks.workunit.client.0.smithi096.stdout:Running main() from gmock_main.cc 2024-03-21T06:35:41.884 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] Running 7 tests from 1 test suite. 2024-03-21T06:35:41.884 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment set-up. 2024-03-21T06:35:41.884 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 7 tests from NeoRadosWriteOps 2024-03-21T06:35:41.884 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosWriteOps.AssertExists 2024-03-21T06:35:44.883 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosWriteOps.AssertExists (3000 ms) 2024-03-21T06:35:44.883 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosWriteOps.AssertVersion 2024-03-21T06:35:47.436 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosWriteOps.AssertVersion (2552 ms) 2024-03-21T06:35:47.436 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosWriteOps.Xattrs 2024-03-21T06:35:50.442 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosWriteOps.Xattrs (3006 ms) 2024-03-21T06:35:50.442 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosWriteOps.Write 2024-03-21T06:35:53.459 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosWriteOps.Write (3017 ms) 2024-03-21T06:35:53.459 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosWriteOps.Exec 2024-03-21T06:35:56.470 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosWriteOps.Exec (3012 ms) 2024-03-21T06:35:56.471 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosWriteOps.WriteSame 2024-03-21T06:35:59.483 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosWriteOps.WriteSame (3013 ms) 2024-03-21T06:35:59.483 INFO:tasks.workunit.client.0.smithi096.stdout:[ RUN ] NeoRadosWriteOps.CmpExt 2024-03-21T06:36:05.870 INFO:tasks.workunit.client.0.smithi096.stdout:[ OK ] NeoRadosWriteOps.CmpExt (6387 ms) 2024-03-21T06:36:05.871 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] 7 tests from NeoRadosWriteOps (23987 ms total) 2024-03-21T06:36:05.871 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-21T06:36:05.871 INFO:tasks.workunit.client.0.smithi096.stdout:[----------] Global test environment tear-down 2024-03-21T06:36:05.871 INFO:tasks.workunit.client.0.smithi096.stdout:[==========] 7 tests from 1 test suite ran. (23987 ms total) 2024-03-21T06:36:05.871 INFO:tasks.workunit.client.0.smithi096.stdout:[ PASSED ] 7 tests. 2024-03-21T06:36:05.873 INFO:tasks.workunit.client.0.smithi096.stderr:+ ret=0 2024-03-21T06:36:05.873 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -eq 1 ']' 2024-03-21T06:36:05.873 INFO:tasks.workunit.client.0.smithi096.stderr:+ exit 0 2024-03-21T06:36:05.873 INFO:tasks.workunit.client.0.smithi096.stderr:+ cleanup 2024-03-21T06:36:05.873 INFO:tasks.workunit.client.0.smithi096.stderr:+ pkill -P 32596 2024-03-21T06:36:06.077 INFO:tasks.workunit.client.0.smithi096.stderr:+ true 2024-03-21T06:36:06.078 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-21T06:36:06.078 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T06:36:06.146 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-03-21T06:36:06.147 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-03-21T06:36:06.148 DEBUG:teuthology.orchestra.run.smithi096:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=63e11116edc7c1585450815ed7030c70ea0ebef9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-03-21T06:36:06.204 INFO:tasks.workunit.client.0.smithi096.stderr:++ uuidgen 2024-03-21T06:36:06.224 INFO:tasks.workunit.client.0.smithi096.stderr:+ p=13dbd6ab-5566-4934-957c-dcb5f3ce8145 2024-03-21T06:36:06.224 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph osd pool create 13dbd6ab-5566-4934-957c-dcb5f3ce8145 12 2024-03-21T06:36:06.310 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.304+0000 7fc89286d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:06.310 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.311+0000 7fc89286d640 1 Processor -- start 2024-03-21T06:36:06.310 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.311+0000 7fc89286d640 1 -- start start 2024-03-21T06:36:06.312 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.312+0000 7fc89286d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc88c09c550 0x7fc88c136420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:06.312 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.312+0000 7fc89286d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88c09bb30 con 0x7fc88c0a16d0 2024-03-21T06:36:06.312 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.312+0000 7fc89286d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc88c06edc0 con 0x7fc88c09c550 2024-03-21T06:36:06.312 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.312+0000 7fc88b7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc88c09c550 0x7fc88c136420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:06.312 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.312+0000 7fc88bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fc88c0a16d0 0x7fc88c09be40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59154/0 (socket says 172.21.15.96:59154) 2024-03-21T06:36:06.312 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.312+0000 7fc88bfff640 1 -- 172.21.15.96:0/1091415798 learned_addr learned my addr 172.21.15.96:0/1091415798 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:36:06.312 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.312+0000 7fc88b7fe640 1 -- 172.21.15.96:0/1091415798 >> v1:172.21.15.96:6789/0 conn(0x7fc88c0a16d0 legacy=0x7fc88c09be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:06.312 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.312+0000 7fc88b7fe640 1 -- 172.21.15.96:0/1091415798 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c06e1c0 con 0x7fc88c09c550 2024-03-21T06:36:06.312 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.313+0000 7fc88b7fe640 1 --2- 172.21.15.96:0/1091415798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc88c09c550 0x7fc88c136420 crc :-1 s=READY pgs=12838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb43dc81fb77a491 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:06.312 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.313+0000 7fc88affd640 1 -- 172.21.15.96:0/1091415798 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc880009ce0 con 0x7fc88c09c550 2024-03-21T06:36:06.313 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.313+0000 7fc88affd640 1 -- 172.21.15.96:0/1091415798 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc880009e40 con 0x7fc88c09c550 2024-03-21T06:36:06.313 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.313+0000 7fc88affd640 1 -- 172.21.15.96:0/1091415798 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc88000e360 con 0x7fc88c09c550 2024-03-21T06:36:06.314 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.313+0000 7fc89286d640 1 -- 172.21.15.96:0/1091415798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc88c09c550 msgr2=0x7fc88c136420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:06.314 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.313+0000 7fc89286d640 1 --2- 172.21.15.96:0/1091415798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc88c09c550 0x7fc88c136420 crc :-1 s=READY pgs=12838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:06.314 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.313+0000 7fc89286d640 1 -- 172.21.15.96:0/1091415798 shutdown_connections 2024-03-21T06:36:06.314 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.313+0000 7fc89286d640 1 --2- 172.21.15.96:0/1091415798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc88c09c550 0x7fc88c136420 unknown :-1 s=CLOSED pgs=12838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:06.314 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.313+0000 7fc89286d640 1 -- 172.21.15.96:0/1091415798 >> 172.21.15.96:0/1091415798 conn(0x7fc88c06c890 msgr2=0x7fc88c06cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:06.314 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.313+0000 7fc89286d640 1 -- 172.21.15.96:0/1091415798 shutdown_connections 2024-03-21T06:36:06.314 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.313+0000 7fc89286d640 1 -- 172.21.15.96:0/1091415798 wait complete. 2024-03-21T06:36:06.314 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.314+0000 7fc89286d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:06.315 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.314+0000 7fc89286d640 1 Processor -- start 2024-03-21T06:36:06.315 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.315+0000 7fc89286d640 1 -- start start 2024-03-21T06:36:06.316 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.315+0000 7fc89286d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc88c09c550 0x7fc88c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:06.317 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.315+0000 7fc89286d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c1222a0 con 0x7fc88c09c550 2024-03-21T06:36:06.317 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.315+0000 7fc88bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc88c09c550 0x7fc88c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:06.317 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.315+0000 7fc88bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc88c09c550 0x7fc88c06f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33128/0 (socket says 172.21.15.96:33128) 2024-03-21T06:36:06.317 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.315+0000 7fc88bfff640 1 -- 172.21.15.96:0/1876192287 learned_addr learned my addr 172.21.15.96:0/1876192287 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:06.317 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.316+0000 7fc88bfff640 1 -- 172.21.15.96:0/1876192287 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c09cc60 con 0x7fc88c09c550 2024-03-21T06:36:06.317 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.316+0000 7fc88bfff640 1 --2- 172.21.15.96:0/1876192287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc88c09c550 0x7fc88c06f0d0 crc :-1 s=READY pgs=12839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:06.317 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.316+0000 7fc888ff9640 1 -- 172.21.15.96:0/1876192287 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc87c0059d0 con 0x7fc88c09c550 2024-03-21T06:36:06.317 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.316+0000 7fc888ff9640 1 -- 172.21.15.96:0/1876192287 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc87c005b30 con 0x7fc88c09c550 2024-03-21T06:36:06.317 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.316+0000 7fc89286d640 1 -- 172.21.15.96:0/1876192287 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88c070c50 con 0x7fc88c09c550 2024-03-21T06:36:06.317 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.316+0000 7fc888ff9640 1 -- 172.21.15.96:0/1876192287 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc87c005dd0 con 0x7fc88c09c550 2024-03-21T06:36:06.317 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.316+0000 7fc89286d640 1 -- 172.21.15.96:0/1876192287 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88c122700 con 0x7fc88c09c550 2024-03-21T06:36:06.317 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.317+0000 7fc888ff9640 1 -- 172.21.15.96:0/1876192287 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fc87c0070b0 con 0x7fc88c09c550 2024-03-21T06:36:06.317 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.317+0000 7fc888ff9640 1 --2- 172.21.15.96:0/1876192287 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc878038260 0x7fc87803a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:06.318 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.317+0000 7fc888ff9640 1 -- 172.21.15.96:0/1876192287 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(681..681 src has 402..681) v4 ==== 3403+0+0 (crc 0 0 0) 0x7fc87c047af0 con 0x7fc88c09c550 2024-03-21T06:36:06.318 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.317+0000 7fc89286d640 1 -- 172.21.15.96:0/1876192287 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88c122950 con 0x7fc88c09c550 2024-03-21T06:36:06.318 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.317+0000 7fc88b7fe640 1 --2- 172.21.15.96:0/1876192287 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc878038260 0x7fc87803a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:06.318 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.318+0000 7fc88b7fe640 1 --2- 172.21.15.96:0/1876192287 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc878038260 0x7fc87803a770 crc :-1 s=READY pgs=3823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:06.321 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.322+0000 7fc888ff9640 1 -- 172.21.15.96:0/1876192287 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fc87c00d6f0 con 0x7fc88c09c550 2024-03-21T06:36:06.496 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.497+0000 7fc89286d640 1 -- 172.21.15.96:0/1876192287 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "pg_num": 12} v 0) v1 -- 0x7fc88c06bd30 con 0x7fc88c09c550 2024-03-21T06:36:06.871 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:06.872+0000 7fc888ff9640 1 -- 172.21.15.96:0/1876192287 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "pg_num": 12}]=0 pool '13dbd6ab-5566-4934-957c-dcb5f3ce8145' created v682) v1 ==== 176+0+0 (crc 0 0 0) 0x7fc87c00d150 con 0x7fc88c09c550 2024-03-21T06:36:07.058 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.058+0000 7fc89286d640 1 -- 172.21.15.96:0/1876192287 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "pg_num": 12} v 0) v1 -- 0x7fc88c09ca80 con 0x7fc88c09c550 2024-03-21T06:36:07.058 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.059+0000 7fc888ff9640 1 -- 172.21.15.96:0/1876192287 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "pg_num": 12}]=0 pool '13dbd6ab-5566-4934-957c-dcb5f3ce8145' already exists v682) v1 ==== 183+0+0 (crc 0 0 0) 0x7fc87c00d330 con 0x7fc88c09c550 2024-03-21T06:36:07.058 INFO:tasks.workunit.client.0.smithi096.stderr:pool '13dbd6ab-5566-4934-957c-dcb5f3ce8145' already exists 2024-03-21T06:36:07.061 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.061+0000 7fc89286d640 1 -- 172.21.15.96:0/1876192287 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc878038260 msgr2=0x7fc87803a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:07.061 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.061+0000 7fc89286d640 1 --2- 172.21.15.96:0/1876192287 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc878038260 0x7fc87803a770 crc :-1 s=READY pgs=3823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:07.061 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.061+0000 7fc89286d640 1 -- 172.21.15.96:0/1876192287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc88c09c550 msgr2=0x7fc88c06f0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:07.061 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.061+0000 7fc89286d640 1 --2- 172.21.15.96:0/1876192287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc88c09c550 0x7fc88c06f0d0 crc :-1 s=READY pgs=12839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:07.061 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.061+0000 7fc89286d640 1 -- 172.21.15.96:0/1876192287 shutdown_connections 2024-03-21T06:36:07.061 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.061+0000 7fc89286d640 1 --2- 172.21.15.96:0/1876192287 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc878038260 0x7fc87803a770 unknown :-1 s=CLOSED pgs=3823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:07.061 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.061+0000 7fc89286d640 1 --2- 172.21.15.96:0/1876192287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc88c09c550 0x7fc88c06f0d0 unknown :-1 s=CLOSED pgs=12839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:07.061 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.061+0000 7fc89286d640 1 -- 172.21.15.96:0/1876192287 >> 172.21.15.96:0/1876192287 conn(0x7fc88c06c890 msgr2=0x7fc88c1b9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:07.061 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.061+0000 7fc89286d640 1 -- 172.21.15.96:0/1876192287 shutdown_connections 2024-03-21T06:36:07.061 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.061+0000 7fc89286d640 1 -- 172.21.15.96:0/1876192287 wait complete. 2024-03-21T06:36:07.071 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph osd pool set-quota 13dbd6ab-5566-4934-957c-dcb5f3ce8145 max_objects 10 2024-03-21T06:36:07.162 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.155+0000 7f068479c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:07.163 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.163+0000 7f068479c640 1 Processor -- start 2024-03-21T06:36:07.163 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.163+0000 7f068479c640 1 -- start start 2024-03-21T06:36:07.166 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.164+0000 7f068479c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f067c0a0550 0x7f067c139750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:07.166 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.164+0000 7f068479c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f067c0a3240 con 0x7f067c0a2ce0 2024-03-21T06:36:07.166 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.164+0000 7f068479c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067c06e660 con 0x7f067c0a0550 2024-03-21T06:36:07.167 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.164+0000 7f0681d37640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f067c0a0550 0x7f067c139750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:07.167 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.164+0000 7f0682538640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f067c0a2ce0 0x7f067c09fe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59156/0 (socket says 172.21.15.96:59156) 2024-03-21T06:36:07.167 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.164+0000 7f0682538640 1 -- 172.21.15.96:0/1814196440 learned_addr learned my addr 172.21.15.96:0/1814196440 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:36:07.167 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.165+0000 7f0681d37640 1 -- 172.21.15.96:0/1814196440 >> v1:172.21.15.96:6789/0 conn(0x7f067c0a2ce0 legacy=0x7f067c09fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:07.167 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.165+0000 7f0681d37640 1 -- 172.21.15.96:0/1814196440 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c06e240 con 0x7f067c0a0550 2024-03-21T06:36:07.167 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.165+0000 7f0681536640 1 -- 172.21.15.96:0/1814196440 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893320260 0 0) 0x7f067c0a3240 con 0x7f067c0a2ce0 2024-03-21T06:36:07.167 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.165+0000 7f0681d37640 1 --2- 172.21.15.96:0/1814196440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f067c0a0550 0x7f067c139750 crc :-1 s=READY pgs=12841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a4afc3518d387c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:07.167 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.165+0000 7f0681536640 1 -- 172.21.15.96:0/1814196440 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f066c009ce0 con 0x7f067c0a0550 2024-03-21T06:36:07.168 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.165+0000 7f0681536640 1 -- 172.21.15.96:0/1814196440 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f066c009e40 con 0x7f067c0a0550 2024-03-21T06:36:07.168 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.165+0000 7f0681536640 1 -- 172.21.15.96:0/1814196440 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f066c00e360 con 0x7f067c0a0550 2024-03-21T06:36:07.168 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.166+0000 7f068479c640 1 -- 172.21.15.96:0/1814196440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f067c0a0550 msgr2=0x7f067c139750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:07.168 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.166+0000 7f068479c640 1 --2- 172.21.15.96:0/1814196440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f067c0a0550 0x7f067c139750 crc :-1 s=READY pgs=12841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:07.168 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.166+0000 7f068479c640 1 -- 172.21.15.96:0/1814196440 shutdown_connections 2024-03-21T06:36:07.168 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.166+0000 7f068479c640 1 --2- 172.21.15.96:0/1814196440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f067c0a0550 0x7f067c139750 unknown :-1 s=CLOSED pgs=12841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:07.168 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.166+0000 7f068479c640 1 -- 172.21.15.96:0/1814196440 >> 172.21.15.96:0/1814196440 conn(0x7f067c06ca30 msgr2=0x7f067c06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:07.169 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.166+0000 7f068479c640 1 -- 172.21.15.96:0/1814196440 shutdown_connections 2024-03-21T06:36:07.169 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.166+0000 7f068479c640 1 -- 172.21.15.96:0/1814196440 wait complete. 2024-03-21T06:36:07.169 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.166+0000 7f068479c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:07.169 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.167+0000 7f068479c640 1 Processor -- start 2024-03-21T06:36:07.169 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.167+0000 7f068479c640 1 -- start start 2024-03-21T06:36:07.173 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.167+0000 7f068479c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f067c0a0550 0x7f067c120e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:07.174 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.167+0000 7f068479c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c134e40 con 0x7f067c0a0550 2024-03-21T06:36:07.174 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.168+0000 7f0682538640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f067c0a0550 0x7f067c120e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:07.174 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.168+0000 7f0682538640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f067c0a0550 0x7f067c120e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33134/0 (socket says 172.21.15.96:33134) 2024-03-21T06:36:07.174 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.168+0000 7f0682538640 1 -- 172.21.15.96:0/80853481 learned_addr learned my addr 172.21.15.96:0/80853481 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:07.174 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.168+0000 7f0682538640 1 -- 172.21.15.96:0/80853481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c1350c0 con 0x7f067c0a0550 2024-03-21T06:36:07.174 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.168+0000 7f0682538640 1 --2- 172.21.15.96:0/80853481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f067c0a0550 0x7f067c120e30 crc :-1 s=READY pgs=12842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:07.175 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.168+0000 7f0672ffd640 1 -- 172.21.15.96:0/80853481 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f06780070b0 con 0x7f067c0a0550 2024-03-21T06:36:07.175 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.168+0000 7f0672ffd640 1 -- 172.21.15.96:0/80853481 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0678007210 con 0x7f067c0a0550 2024-03-21T06:36:07.175 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.168+0000 7f0672ffd640 1 -- 172.21.15.96:0/80853481 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f06780074b0 con 0x7f067c0a0550 2024-03-21T06:36:07.175 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.168+0000 7f068479c640 1 -- 172.21.15.96:0/80853481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067c124400 con 0x7f067c0a0550 2024-03-21T06:36:07.175 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.168+0000 7f068479c640 1 -- 172.21.15.96:0/80853481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067c06a4c0 con 0x7f067c0a0550 2024-03-21T06:36:07.175 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.169+0000 7f0672ffd640 1 -- 172.21.15.96:0/80853481 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f0678007660 con 0x7f067c0a0550 2024-03-21T06:36:07.176 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.169+0000 7f068479c640 1 -- 172.21.15.96:0/80853481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067c124ea0 con 0x7f067c0a0550 2024-03-21T06:36:07.176 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.169+0000 7f0672ffd640 1 --2- 172.21.15.96:0/80853481 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f0660038260 0x7f066003a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:07.176 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.169+0000 7f0672ffd640 1 -- 172.21.15.96:0/80853481 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(682..682 src has 402..682) v4 ==== 3778+0+0 (crc 0 0 0) 0x7f0678048890 con 0x7f067c0a0550 2024-03-21T06:36:07.176 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.170+0000 7f0681d37640 1 --2- 172.21.15.96:0/80853481 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f0660038260 0x7f066003a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:07.176 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.170+0000 7f0681d37640 1 --2- 172.21.15.96:0/80853481 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f0660038260 0x7f066003a770 crc :-1 s=READY pgs=3824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:07.176 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.174+0000 7f0672ffd640 1 -- 172.21.15.96:0/80853481 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f06780094c0 con 0x7f067c0a0550 2024-03-21T06:36:07.348 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.348+0000 7f068479c640 1 -- 172.21.15.96:0/80853481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f067c06fe00 con 0x7f067c0a0550 2024-03-21T06:36:07.436 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.437+0000 7f0672ffd640 1 -- 172.21.15.96:0/80853481 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 v683) v1 ==== 223+0+0 (crc 0 0 0) 0x7f06780090b0 con 0x7f067c0a0550 2024-03-21T06:36:07.580 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:07.581+0000 7f068479c640 1 -- 172.21.15.96:0/80853481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f067c068e80 con 0x7f067c0a0550 2024-03-21T06:36:08.443 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.443+0000 7f0672ffd640 1 -- 172.21.15.96:0/80853481 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 v684) v1 ==== 223+0+0 (crc 0 0 0) 0x7f0678009290 con 0x7f067c0a0550 2024-03-21T06:36:08.443 INFO:tasks.workunit.client.0.smithi096.stderr:set-quota max_objects = 10 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 2024-03-21T06:36:08.445 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.445+0000 7f068479c640 1 -- 172.21.15.96:0/80853481 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f0660038260 msgr2=0x7f066003a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:08.445 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.445+0000 7f068479c640 1 --2- 172.21.15.96:0/80853481 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f0660038260 0x7f066003a770 crc :-1 s=READY pgs=3824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:08.445 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.445+0000 7f068479c640 1 -- 172.21.15.96:0/80853481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f067c0a0550 msgr2=0x7f067c120e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:08.445 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.445+0000 7f068479c640 1 --2- 172.21.15.96:0/80853481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f067c0a0550 0x7f067c120e30 crc :-1 s=READY pgs=12842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:08.445 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.446+0000 7f068479c640 1 -- 172.21.15.96:0/80853481 shutdown_connections 2024-03-21T06:36:08.445 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.446+0000 7f068479c640 1 --2- 172.21.15.96:0/80853481 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f0660038260 0x7f066003a770 unknown :-1 s=CLOSED pgs=3824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:08.445 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.446+0000 7f068479c640 1 --2- 172.21.15.96:0/80853481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f067c0a0550 0x7f067c120e30 unknown :-1 s=CLOSED pgs=12842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:08.446 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.446+0000 7f068479c640 1 -- 172.21.15.96:0/80853481 >> 172.21.15.96:0/80853481 conn(0x7f067c06ca30 msgr2=0x7f067c1ba230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:08.446 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.446+0000 7f068479c640 1 -- 172.21.15.96:0/80853481 shutdown_connections 2024-03-21T06:36:08.446 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.446+0000 7f068479c640 1 -- 172.21.15.96:0/80853481 wait complete. 2024-03-21T06:36:08.454 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph osd pool application enable 13dbd6ab-5566-4934-957c-dcb5f3ce8145 rados 2024-03-21T06:36:08.544 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.537+0000 7fa3d5d9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:08.544 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.545+0000 7fa3d5d9c640 1 Processor -- start 2024-03-21T06:36:08.544 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.545+0000 7fa3d5d9c640 1 -- start start 2024-03-21T06:36:08.546 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.546+0000 7fa3d5d9c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa3d01325e0 0x7fa3d01329f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:08.546 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.546+0000 7fa3d5d9c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d0131f80 con 0x7fa3d009ca30 2024-03-21T06:36:08.546 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.546+0000 7fa3d5d9c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d0132f30 con 0x7fa3d01325e0 2024-03-21T06:36:08.546 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.546+0000 7fa3cf7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa3d01325e0 0x7fa3d01329f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:08.546 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.546+0000 7fa3ceffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fa3d009ca30 0x7fa3d009ce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59168/0 (socket says 172.21.15.96:59168) 2024-03-21T06:36:08.546 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.546+0000 7fa3ceffd640 1 -- 172.21.15.96:0/682489064 learned_addr learned my addr 172.21.15.96:0/682489064 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:36:08.546 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.546+0000 7fa3cf7fe640 1 -- 172.21.15.96:0/682489064 >> v1:172.21.15.96:6789/0 conn(0x7fa3d009ca30 legacy=0x7fa3d009ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:08.546 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.546+0000 7fa3cf7fe640 1 -- 172.21.15.96:0/682489064 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d0067f30 con 0x7fa3d01325e0 2024-03-21T06:36:08.547 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.546+0000 7fa3ce7fc640 1 -- 172.21.15.96:0/682489064 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281163723 0 0) 0x7fa3d0131f80 con 0x7fa3d009ca30 2024-03-21T06:36:08.547 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.547+0000 7fa3cf7fe640 1 --2- 172.21.15.96:0/682489064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d01325e0 0x7fa3d01329f0 crc :-1 s=READY pgs=12844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c84b8a369018ab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:08.547 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.547+0000 7fa3ce7fc640 1 -- 172.21.15.96:0/682489064 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa3c4009ce0 con 0x7fa3d01325e0 2024-03-21T06:36:08.547 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.547+0000 7fa3ce7fc640 1 -- 172.21.15.96:0/682489064 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa3c4009e40 con 0x7fa3d01325e0 2024-03-21T06:36:08.547 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.547+0000 7fa3ce7fc640 1 -- 172.21.15.96:0/682489064 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa3c400e360 con 0x7fa3d01325e0 2024-03-21T06:36:08.548 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.547+0000 7fa3d5d9c640 1 -- 172.21.15.96:0/682489064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d01325e0 msgr2=0x7fa3d01329f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:08.548 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.547+0000 7fa3d5d9c640 1 --2- 172.21.15.96:0/682489064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d01325e0 0x7fa3d01329f0 crc :-1 s=READY pgs=12844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:08.548 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.547+0000 7fa3d5d9c640 1 -- 172.21.15.96:0/682489064 shutdown_connections 2024-03-21T06:36:08.548 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.547+0000 7fa3d5d9c640 1 --2- 172.21.15.96:0/682489064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d01325e0 0x7fa3d01329f0 unknown :-1 s=CLOSED pgs=12844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:08.549 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.547+0000 7fa3d5d9c640 1 -- 172.21.15.96:0/682489064 >> 172.21.15.96:0/682489064 conn(0x7fa3d00a05c0 msgr2=0x7fa3d00a09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:08.549 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.547+0000 7fa3d5d9c640 1 -- 172.21.15.96:0/682489064 shutdown_connections 2024-03-21T06:36:08.549 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.547+0000 7fa3d5d9c640 1 -- 172.21.15.96:0/682489064 wait complete. 2024-03-21T06:36:08.549 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.548+0000 7fa3d5d9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:08.549 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.549+0000 7fa3d5d9c640 1 Processor -- start 2024-03-21T06:36:08.549 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.549+0000 7fa3d5d9c640 1 -- start start 2024-03-21T06:36:08.551 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.550+0000 7fa3d5d9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d009ca30 0x7fa3d011db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:08.551 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.550+0000 7fa3d5d9c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d0121c00 con 0x7fa3d009ca30 2024-03-21T06:36:08.551 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.550+0000 7fa3cf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d009ca30 0x7fa3d011db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:08.551 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.550+0000 7fa3cf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d009ca30 0x7fa3d011db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33152/0 (socket says 172.21.15.96:33152) 2024-03-21T06:36:08.551 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.550+0000 7fa3cf7fe640 1 -- 172.21.15.96:0/1116331033 learned_addr learned my addr 172.21.15.96:0/1116331033 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:08.551 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.550+0000 7fa3cf7fe640 1 -- 172.21.15.96:0/1116331033 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d011ebc0 con 0x7fa3d009ca30 2024-03-21T06:36:08.551 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.550+0000 7fa3cf7fe640 1 --2- 172.21.15.96:0/1116331033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d009ca30 0x7fa3d011db10 crc :-1 s=READY pgs=12845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:08.551 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.550+0000 7fa3b7fff640 1 -- 172.21.15.96:0/1116331033 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa3c4014540 con 0x7fa3d009ca30 2024-03-21T06:36:08.551 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.550+0000 7fa3b7fff640 1 -- 172.21.15.96:0/1116331033 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa3c40146a0 con 0x7fa3d009ca30 2024-03-21T06:36:08.551 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.550+0000 7fa3d5d9c640 1 -- 172.21.15.96:0/1116331033 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d0120790 con 0x7fa3d009ca30 2024-03-21T06:36:08.552 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.550+0000 7fa3b7fff640 1 -- 172.21.15.96:0/1116331033 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa3c4014940 con 0x7fa3d009ca30 2024-03-21T06:36:08.552 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.550+0000 7fa3d5d9c640 1 -- 172.21.15.96:0/1116331033 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d0120f40 con 0x7fa3d009ca30 2024-03-21T06:36:08.552 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.551+0000 7fa3b7fff640 1 -- 172.21.15.96:0/1116331033 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fa3c4014af0 con 0x7fa3d009ca30 2024-03-21T06:36:08.552 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.552+0000 7fa3b7fff640 1 --2- 172.21.15.96:0/1116331033 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa3bc038260 0x7fa3bc03a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:08.552 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.552+0000 7fa3b7fff640 1 -- 172.21.15.96:0/1116331033 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(684..684 src has 402..684) v4 ==== 3778+0+0 (crc 0 0 0) 0x7fa3c4019040 con 0x7fa3d009ca30 2024-03-21T06:36:08.552 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.552+0000 7fa3d5d9c640 1 -- 172.21.15.96:0/1116331033 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d0067f30 con 0x7fa3d009ca30 2024-03-21T06:36:08.552 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.552+0000 7fa3ceffd640 1 --2- 172.21.15.96:0/1116331033 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa3bc038260 0x7fa3bc03a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:08.553 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.552+0000 7fa3ceffd640 1 --2- 172.21.15.96:0/1116331033 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa3bc038260 0x7fa3bc03a770 crc :-1 s=READY pgs=3825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:08.556 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.557+0000 7fa3b7fff640 1 -- 172.21.15.96:0/1116331033 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fa3d0067f30 con 0x7fa3d009ca30 2024-03-21T06:36:08.738 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:08.739+0000 7fa3d5d9c640 1 -- 172.21.15.96:0/1116331033 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "app": "rados"} v 0) v1 -- 0x7fa3d0067f30 con 0x7fa3d009ca30 2024-03-21T06:36:09.445 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:09.446+0000 7fa3b7fff640 1 -- 172.21.15.96:0/1116331033 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "app": "rados"}]=0 enabled application 'rados' on pool '13dbd6ab-5566-4934-957c-dcb5f3ce8145' v685) v1 ==== 213+0+0 (crc 0 0 0) 0x7fa3d0067f30 con 0x7fa3d009ca30 2024-03-21T06:36:09.576 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:09.577+0000 7fa3d5d9c640 1 -- 172.21.15.96:0/1116331033 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "app": "rados"} v 0) v1 -- 0x7fa3d0067f30 con 0x7fa3d009ca30 2024-03-21T06:36:10.446 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.447+0000 7fa3b7fff640 1 -- 172.21.15.96:0/1116331033 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "app": "rados"}]=0 enabled application 'rados' on pool '13dbd6ab-5566-4934-957c-dcb5f3ce8145' v686) v1 ==== 213+0+0 (crc 0 0 0) 0x7fa3d0067f30 con 0x7fa3d009ca30 2024-03-21T06:36:10.447 INFO:tasks.workunit.client.0.smithi096.stderr:enabled application 'rados' on pool '13dbd6ab-5566-4934-957c-dcb5f3ce8145' 2024-03-21T06:36:10.449 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.450+0000 7fa3d5d9c640 1 -- 172.21.15.96:0/1116331033 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa3bc038260 msgr2=0x7fa3bc03a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.449 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.450+0000 7fa3d5d9c640 1 --2- 172.21.15.96:0/1116331033 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa3bc038260 0x7fa3bc03a770 crc :-1 s=READY pgs=3825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.449 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.450+0000 7fa3d5d9c640 1 -- 172.21.15.96:0/1116331033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d009ca30 msgr2=0x7fa3d011db10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.449 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.450+0000 7fa3d5d9c640 1 --2- 172.21.15.96:0/1116331033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d009ca30 0x7fa3d011db10 crc :-1 s=READY pgs=12845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.450 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.450+0000 7fa3d5d9c640 1 -- 172.21.15.96:0/1116331033 shutdown_connections 2024-03-21T06:36:10.450 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.450+0000 7fa3d5d9c640 1 --2- 172.21.15.96:0/1116331033 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa3bc038260 0x7fa3bc03a770 unknown :-1 s=CLOSED pgs=3825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.450 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.450+0000 7fa3d5d9c640 1 --2- 172.21.15.96:0/1116331033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3d009ca30 0x7fa3d011db10 unknown :-1 s=CLOSED pgs=12845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.450 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.450+0000 7fa3d5d9c640 1 -- 172.21.15.96:0/1116331033 >> 172.21.15.96:0/1116331033 conn(0x7fa3d00a05c0 msgr2=0x7fa3d0070e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:10.450 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.450+0000 7fa3d5d9c640 1 -- 172.21.15.96:0/1116331033 shutdown_connections 2024-03-21T06:36:10.450 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.450+0000 7fa3d5d9c640 1 -- 172.21.15.96:0/1116331033 wait complete. 2024-03-21T06:36:10.465 INFO:tasks.workunit.client.0.smithi096.stderr:++ seq 1 10 2024-03-21T06:36:10.491 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:36:10.492 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 13dbd6ab-5566-4934-957c-dcb5f3ce8145 put obj1 /etc/passwd 2024-03-21T06:36:10.525 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.526+0000 7fa91727ae00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:10.531 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.532+0000 7fa91727ae00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:10.531 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.532+0000 7fa91727ae00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:10.531 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.532+0000 7fa91727ae00 1 Processor -- start 2024-03-21T06:36:10.531 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.532+0000 7fa91727ae00 1 -- start start 2024-03-21T06:36:10.533 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.533+0000 7fa91727ae00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed5088d7e0 0x55ed509ad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:10.533 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.533+0000 7fa91727ae00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed5082cea0 con 0x55ed5088d7e0 2024-03-21T06:36:10.533 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.533+0000 7fa90f79a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed5088d7e0 0x55ed509ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:10.533 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.533+0000 7fa90f79a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed5088d7e0 0x55ed509ad960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33170/0 (socket says 172.21.15.96:33170) 2024-03-21T06:36:10.533 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.533+0000 7fa90f79a640 1 -- 172.21.15.96:0/798041244 learned_addr learned my addr 172.21.15.96:0/798041244 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:10.533 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.533+0000 7fa90f79a640 1 -- 172.21.15.96:0/798041244 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed507cfa30 con 0x55ed5088d7e0 2024-03-21T06:36:10.533 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.534+0000 7fa90f79a640 1 --2- 172.21.15.96:0/798041244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed5088d7e0 0x55ed509ad960 crc :-1 s=READY pgs=12848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fd1357441029c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:10.533 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.534+0000 7fa90df97640 1 -- 172.21.15.96:0/798041244 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa90400b5d0 con 0x55ed5088d7e0 2024-03-21T06:36:10.533 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.534+0000 7fa90df97640 1 -- 172.21.15.96:0/798041244 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa90400b730 con 0x55ed5088d7e0 2024-03-21T06:36:10.533 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.534+0000 7fa90df97640 1 -- 172.21.15.96:0/798041244 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa90400b9d0 con 0x55ed5088d7e0 2024-03-21T06:36:10.535 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.535+0000 7fa91727ae00 1 -- 172.21.15.96:0/798041244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed5088d7e0 msgr2=0x55ed509ad960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.535 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.535+0000 7fa91727ae00 1 --2- 172.21.15.96:0/798041244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed5088d7e0 0x55ed509ad960 crc :-1 s=READY pgs=12848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.535 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.535+0000 7fa91727ae00 1 -- 172.21.15.96:0/798041244 shutdown_connections 2024-03-21T06:36:10.535 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.535+0000 7fa91727ae00 1 --2- 172.21.15.96:0/798041244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed5088d7e0 0x55ed509ad960 unknown :-1 s=CLOSED pgs=12848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.535 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.535+0000 7fa91727ae00 1 -- 172.21.15.96:0/798041244 >> 172.21.15.96:0/798041244 conn(0x55ed507a1e00 msgr2=0x55ed50921240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:10.535 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.535+0000 7fa91727ae00 1 -- 172.21.15.96:0/798041244 shutdown_connections 2024-03-21T06:36:10.535 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.535+0000 7fa91727ae00 1 -- 172.21.15.96:0/798041244 wait complete. 2024-03-21T06:36:10.535 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.535+0000 7fa91727ae00 1 Processor -- start 2024-03-21T06:36:10.536 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.536+0000 7fa91727ae00 1 -- start start 2024-03-21T06:36:10.537 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.536+0000 7fa91727ae00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed5088d7e0 0x55ed5088fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:10.537 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.536+0000 7fa91727ae00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed507cd550 con 0x55ed5088d7e0 2024-03-21T06:36:10.537 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.537+0000 7fa90f79a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed5088d7e0 0x55ed5088fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:10.537 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.537+0000 7fa90f79a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed5088d7e0 0x55ed5088fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33186/0 (socket says 172.21.15.96:33186) 2024-03-21T06:36:10.537 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.537+0000 7fa90f79a640 1 -- 172.21.15.96:0/1806713205 learned_addr learned my addr 172.21.15.96:0/1806713205 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:10.537 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.537+0000 7fa90f79a640 1 -- 172.21.15.96:0/1806713205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed507c2680 con 0x55ed5088d7e0 2024-03-21T06:36:10.537 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.537+0000 7fa90f79a640 1 --2- 172.21.15.96:0/1806713205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed5088d7e0 0x55ed5088fd60 crc :-1 s=READY pgs=12849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:10.537 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.537+0000 7fa90cf95640 1 -- 172.21.15.96:0/1806713205 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa904012a10 con 0x55ed5088d7e0 2024-03-21T06:36:10.538 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.537+0000 7fa90cf95640 1 -- 172.21.15.96:0/1806713205 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa904005040 con 0x55ed5088d7e0 2024-03-21T06:36:10.538 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.537+0000 7fa91727ae00 1 -- 172.21.15.96:0/1806713205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ed507c20e0 con 0x55ed5088d7e0 2024-03-21T06:36:10.538 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.538+0000 7fa90cf95640 1 -- 172.21.15.96:0/1806713205 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa904012ca0 con 0x55ed5088d7e0 2024-03-21T06:36:10.538 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.538+0000 7fa91727ae00 1 -- 172.21.15.96:0/1806713205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed508929a0 con 0x55ed5088d7e0 2024-03-21T06:36:10.539 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.539+0000 7fa90cf95640 1 -- 172.21.15.96:0/1806713205 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fa90400d040 con 0x55ed5088d7e0 2024-03-21T06:36:10.539 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.539+0000 7fa90cf95640 1 --2- 172.21.15.96:0/1806713205 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa8f0036b80 0x7fa8f0039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:10.539 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.539+0000 7fa90cf95640 1 -- 172.21.15.96:0/1806713205 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(686..686 src has 402..686) v4 ==== 3791+0+0 (crc 0 0 0) 0x7fa90400a040 con 0x55ed5088d7e0 2024-03-21T06:36:10.539 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.539+0000 7fa91727ae00 1 --2- 172.21.15.96:0/1806713205 >> v2:172.21.15.184:6811/276480837 conn(0x55ed5088dcf0 0x55ed50dbfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:10.539 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.539+0000 7fa91727ae00 1 -- 172.21.15.96:0/1806713205 --> v2:172.21.15.184:6811/276480837 -- osd_op(unknown.0.0:1 200.7 200:ff7b1f36:::obj1:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) v9 -- 0x55ed5090fa50 con 0x55ed5088dcf0 2024-03-21T06:36:10.539 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.539+0000 7fa90ef99640 1 --2- 172.21.15.96:0/1806713205 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa8f0036b80 0x7fa8f0039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:10.539 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.540+0000 7fa90ff9b640 1 --2- 172.21.15.96:0/1806713205 >> v2:172.21.15.184:6811/276480837 conn(0x55ed5088dcf0 0x55ed50dbfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:36:10.539 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.540+0000 7fa90ef99640 1 --2- 172.21.15.96:0/1806713205 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa8f0036b80 0x7fa8f0039090 crc :-1 s=READY pgs=3826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:10.541 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.542+0000 7fa90ff9b640 1 --2- 172.21.15.96:0/1806713205 >> v2:172.21.15.184:6811/276480837 conn(0x55ed5088dcf0 0x55ed50dbfc80 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:10.758 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.758+0000 7fa90ff9b640 1 -- 172.21.15.96:0/1806713205 <== osd.3 v2:172.21.15.184:6811/276480837 1 ==== osd_op_reply(1 obj1 [writefull 0~11812] v686'1 uv1 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fa90000a430 con 0x55ed5088dcf0 2024-03-21T06:36:10.758 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.758+0000 7fa91727ae00 1 -- 172.21.15.96:0/1806713205 >> v2:172.21.15.184:6811/276480837 conn(0x55ed5088dcf0 msgr2=0x55ed50dbfc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.758 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.758+0000 7fa91727ae00 1 --2- 172.21.15.96:0/1806713205 >> v2:172.21.15.184:6811/276480837 conn(0x55ed5088dcf0 0x55ed50dbfc80 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.758 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.758+0000 7fa91727ae00 1 -- 172.21.15.96:0/1806713205 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa8f0036b80 msgr2=0x7fa8f0039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.758 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.758+0000 7fa91727ae00 1 --2- 172.21.15.96:0/1806713205 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa8f0036b80 0x7fa8f0039090 crc :-1 s=READY pgs=3826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.758 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.758+0000 7fa91727ae00 1 -- 172.21.15.96:0/1806713205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed5088d7e0 msgr2=0x55ed5088fd60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.758 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.758+0000 7fa91727ae00 1 --2- 172.21.15.96:0/1806713205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed5088d7e0 0x55ed5088fd60 crc :-1 s=READY pgs=12849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.758 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.758+0000 7fa91727ae00 1 -- 172.21.15.96:0/1806713205 shutdown_connections 2024-03-21T06:36:10.758 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.758+0000 7fa91727ae00 1 --2- 172.21.15.96:0/1806713205 >> v2:172.21.15.184:6811/276480837 conn(0x55ed5088dcf0 0x55ed50dbfc80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.758 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.758+0000 7fa91727ae00 1 --2- 172.21.15.96:0/1806713205 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa8f0036b80 0x7fa8f0039090 unknown :-1 s=CLOSED pgs=3826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.759 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.758+0000 7fa91727ae00 1 --2- 172.21.15.96:0/1806713205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed5088d7e0 0x55ed5088fd60 unknown :-1 s=CLOSED pgs=12849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.759 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.758+0000 7fa91727ae00 1 -- 172.21.15.96:0/1806713205 >> 172.21.15.96:0/1806713205 conn(0x55ed507a1e00 msgr2=0x55ed50898f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:10.759 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.758+0000 7fa91727ae00 1 -- 172.21.15.96:0/1806713205 shutdown_connections 2024-03-21T06:36:10.759 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.758+0000 7fa91727ae00 1 -- 172.21.15.96:0/1806713205 wait complete. 2024-03-21T06:36:10.759 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:36:10.760 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 13dbd6ab-5566-4934-957c-dcb5f3ce8145 put obj2 /etc/passwd 2024-03-21T06:36:10.788 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.789+0000 7fb52a568e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:10.792 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.793+0000 7fb52a568e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:10.792 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.793+0000 7fb52a568e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:10.793 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.794+0000 7fb52a568e00 1 Processor -- start 2024-03-21T06:36:10.793 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.794+0000 7fb52a568e00 1 -- start start 2024-03-21T06:36:10.798 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.794+0000 7fb52a568e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1464dddf0 0x55e14656a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:10.798 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.794+0000 7fb52a568e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1463e3ea0 con 0x55e1464dddf0 2024-03-21T06:36:10.798 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.794+0000 7fb51f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1464dddf0 0x55e14656a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:10.798 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.794+0000 7fb51f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1464dddf0 0x55e14656a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33204/0 (socket says 172.21.15.96:33204) 2024-03-21T06:36:10.798 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.794+0000 7fb51f7fe640 1 -- 172.21.15.96:0/3962606307 learned_addr learned my addr 172.21.15.96:0/3962606307 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:10.798 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.795+0000 7fb51f7fe640 1 -- 172.21.15.96:0/3962606307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e146386a30 con 0x55e1464dddf0 2024-03-21T06:36:10.798 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.795+0000 7fb51f7fe640 1 --2- 172.21.15.96:0/3962606307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1464dddf0 0x55e14656a530 crc :-1 s=READY pgs=12852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb52b92c1aea6d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:10.798 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.795+0000 7fb51e7fc640 1 -- 172.21.15.96:0/3962606307 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb51000c490 con 0x55e1464dddf0 2024-03-21T06:36:10.798 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.795+0000 7fb51e7fc640 1 -- 172.21.15.96:0/3962606307 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb51000c5f0 con 0x55e1464dddf0 2024-03-21T06:36:10.798 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.795+0000 7fb51e7fc640 1 -- 172.21.15.96:0/3962606307 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb51000c890 con 0x55e1464dddf0 2024-03-21T06:36:10.798 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.795+0000 7fb52a568e00 1 -- 172.21.15.96:0/3962606307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1464dddf0 msgr2=0x55e14656a530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.799 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.795+0000 7fb52a568e00 1 --2- 172.21.15.96:0/3962606307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1464dddf0 0x55e14656a530 crc :-1 s=READY pgs=12852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.799 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.796+0000 7fb52a568e00 1 -- 172.21.15.96:0/3962606307 shutdown_connections 2024-03-21T06:36:10.799 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.796+0000 7fb52a568e00 1 --2- 172.21.15.96:0/3962606307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1464dddf0 0x55e14656a530 unknown :-1 s=CLOSED pgs=12852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.799 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.796+0000 7fb52a568e00 1 -- 172.21.15.96:0/3962606307 >> 172.21.15.96:0/3962606307 conn(0x55e146358e00 msgr2=0x55e146445300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:10.799 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.796+0000 7fb52a568e00 1 -- 172.21.15.96:0/3962606307 shutdown_connections 2024-03-21T06:36:10.800 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.796+0000 7fb52a568e00 1 -- 172.21.15.96:0/3962606307 wait complete. 2024-03-21T06:36:10.800 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.796+0000 7fb52a568e00 1 Processor -- start 2024-03-21T06:36:10.800 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.796+0000 7fb52a568e00 1 -- start start 2024-03-21T06:36:10.800 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.797+0000 7fb52a568e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1464dddf0 0x55e146572750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:10.800 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.797+0000 7fb52a568e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e146384550 con 0x55e1464dddf0 2024-03-21T06:36:10.800 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.797+0000 7fb51f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1464dddf0 0x55e146572750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:10.800 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.797+0000 7fb51f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1464dddf0 0x55e146572750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33208/0 (socket says 172.21.15.96:33208) 2024-03-21T06:36:10.800 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.797+0000 7fb51f7fe640 1 -- 172.21.15.96:0/522097508 learned_addr learned my addr 172.21.15.96:0/522097508 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:10.800 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.797+0000 7fb51f7fe640 1 -- 172.21.15.96:0/522097508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e146379680 con 0x55e1464dddf0 2024-03-21T06:36:10.800 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.797+0000 7fb51f7fe640 1 --2- 172.21.15.96:0/522097508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1464dddf0 0x55e146572750 crc :-1 s=READY pgs=12853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:10.801 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.797+0000 7fb51d7fa640 1 -- 172.21.15.96:0/522097508 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb5100111e0 con 0x55e1464dddf0 2024-03-21T06:36:10.801 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.797+0000 7fb51d7fa640 1 -- 172.21.15.96:0/522097508 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb510011980 con 0x55e1464dddf0 2024-03-21T06:36:10.801 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.797+0000 7fb51d7fa640 1 -- 172.21.15.96:0/522097508 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb510011af0 con 0x55e1464dddf0 2024-03-21T06:36:10.801 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.797+0000 7fb52a568e00 1 -- 172.21.15.96:0/522097508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e1463790e0 con 0x55e1464dddf0 2024-03-21T06:36:10.801 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.797+0000 7fb52a568e00 1 -- 172.21.15.96:0/522097508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1465672f0 con 0x55e1464dddf0 2024-03-21T06:36:10.801 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.798+0000 7fb51d7fa640 1 -- 172.21.15.96:0/522097508 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fb510011ca0 con 0x55e1464dddf0 2024-03-21T06:36:10.801 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.798+0000 7fb51d7fa640 1 --2- 172.21.15.96:0/522097508 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb500036b80 0x7fb500039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:10.801 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.798+0000 7fb51d7fa640 1 -- 172.21.15.96:0/522097508 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(686..686 src has 402..686) v4 ==== 3791+0+0 (crc 0 0 0) 0x7fb5100482f0 con 0x55e1464dddf0 2024-03-21T06:36:10.801 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.799+0000 7fb52a568e00 1 --2- 172.21.15.96:0/522097508 >> v2:172.21.15.184:6811/276480837 conn(0x55e1464e0d40 0x55e1464dc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:10.801 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.799+0000 7fb52a568e00 1 -- 172.21.15.96:0/522097508 --> v2:172.21.15.184:6811/276480837 -- osd_op(unknown.0.0:1 200.8 200:104778fc:::obj2:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) v9 -- 0x55e1464c7210 con 0x55e1464e0d40 2024-03-21T06:36:10.801 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.799+0000 7fb517fff640 1 --2- 172.21.15.96:0/522097508 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb500036b80 0x7fb500039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:10.802 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.799+0000 7fb51ffff640 1 --2- 172.21.15.96:0/522097508 >> v2:172.21.15.184:6811/276480837 conn(0x55e1464e0d40 0x55e1464dc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:36:10.802 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.799+0000 7fb517fff640 1 --2- 172.21.15.96:0/522097508 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb500036b80 0x7fb500039090 crc :-1 s=READY pgs=3827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:10.802 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.801+0000 7fb51ffff640 1 --2- 172.21.15.96:0/522097508 >> v2:172.21.15.184:6811/276480837 conn(0x55e1464e0d40 0x55e1464dc520 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:10.807 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.807+0000 7fb51ffff640 1 -- 172.21.15.96:0/522097508 <== osd.3 v2:172.21.15.184:6811/276480837 1 ==== osd_op_reply(1 obj2 [writefull 0~11812] v686'1 uv1 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb51800a430 con 0x55e1464e0d40 2024-03-21T06:36:10.808 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.807+0000 7fb52a568e00 1 -- 172.21.15.96:0/522097508 >> v2:172.21.15.184:6811/276480837 conn(0x55e1464e0d40 msgr2=0x55e1464dc520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.808 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.807+0000 7fb52a568e00 1 --2- 172.21.15.96:0/522097508 >> v2:172.21.15.184:6811/276480837 conn(0x55e1464e0d40 0x55e1464dc520 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.808 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.807+0000 7fb52a568e00 1 -- 172.21.15.96:0/522097508 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb500036b80 msgr2=0x7fb500039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.808 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.807+0000 7fb52a568e00 1 --2- 172.21.15.96:0/522097508 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb500036b80 0x7fb500039090 crc :-1 s=READY pgs=3827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.808 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.807+0000 7fb52a568e00 1 -- 172.21.15.96:0/522097508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1464dddf0 msgr2=0x55e146572750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.808 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.807+0000 7fb52a568e00 1 --2- 172.21.15.96:0/522097508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1464dddf0 0x55e146572750 crc :-1 s=READY pgs=12853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.808 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.807+0000 7fb52a568e00 1 -- 172.21.15.96:0/522097508 shutdown_connections 2024-03-21T06:36:10.808 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.807+0000 7fb52a568e00 1 --2- 172.21.15.96:0/522097508 >> v2:172.21.15.184:6811/276480837 conn(0x55e1464e0d40 0x55e1464dc520 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.808 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.807+0000 7fb52a568e00 1 --2- 172.21.15.96:0/522097508 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb500036b80 0x7fb500039090 unknown :-1 s=CLOSED pgs=3827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.808 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.807+0000 7fb52a568e00 1 --2- 172.21.15.96:0/522097508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1464dddf0 0x55e146572750 unknown :-1 s=CLOSED pgs=12853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.808 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.807+0000 7fb52a568e00 1 -- 172.21.15.96:0/522097508 >> 172.21.15.96:0/522097508 conn(0x55e146358e00 msgr2=0x55e146445300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:10.808 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.807+0000 7fb52a568e00 1 -- 172.21.15.96:0/522097508 shutdown_connections 2024-03-21T06:36:10.808 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.807+0000 7fb52a568e00 1 -- 172.21.15.96:0/522097508 wait complete. 2024-03-21T06:36:10.809 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:36:10.809 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 13dbd6ab-5566-4934-957c-dcb5f3ce8145 put obj3 /etc/passwd 2024-03-21T06:36:10.839 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.840+0000 7f54b506ee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:10.843 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.844+0000 7f54b506ee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:10.844 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.844+0000 7f54b506ee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:10.844 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.844+0000 7f54b506ee00 1 Processor -- start 2024-03-21T06:36:10.844 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.844+0000 7f54b506ee00 1 -- start start 2024-03-21T06:36:10.846 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.845+0000 7f54b506ee00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b0cb50df0 0x564b0cb39d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:10.846 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.845+0000 7f54b506ee00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b0ca56ea0 con 0x564b0cb50df0 2024-03-21T06:36:10.846 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.845+0000 7f54ad59a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b0cb50df0 0x564b0cb39d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:10.846 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.845+0000 7f54ad59a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b0cb50df0 0x564b0cb39d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33218/0 (socket says 172.21.15.96:33218) 2024-03-21T06:36:10.846 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.845+0000 7f54ad59a640 1 -- 172.21.15.96:0/3860983600 learned_addr learned my addr 172.21.15.96:0/3860983600 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:10.846 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.845+0000 7f54ad59a640 1 -- 172.21.15.96:0/3860983600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b0c9f9a30 con 0x564b0cb50df0 2024-03-21T06:36:10.846 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.845+0000 7f54ad59a640 1 --2- 172.21.15.96:0/3860983600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b0cb50df0 0x564b0cb39d30 crc :-1 s=READY pgs=12856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3270caad4c5215b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:10.847 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.846+0000 7f549f7fe640 1 -- 172.21.15.96:0/3860983600 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f54a400b500 con 0x564b0cb50df0 2024-03-21T06:36:10.847 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.846+0000 7f549f7fe640 1 -- 172.21.15.96:0/3860983600 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f54a400b660 con 0x564b0cb50df0 2024-03-21T06:36:10.847 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.846+0000 7f549f7fe640 1 -- 172.21.15.96:0/3860983600 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f54a400b900 con 0x564b0cb50df0 2024-03-21T06:36:10.847 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.846+0000 7f54b506ee00 1 -- 172.21.15.96:0/3860983600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b0cb50df0 msgr2=0x564b0cb39d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.847 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.846+0000 7f54b506ee00 1 --2- 172.21.15.96:0/3860983600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b0cb50df0 0x564b0cb39d30 crc :-1 s=READY pgs=12856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.847 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.846+0000 7f54b506ee00 1 -- 172.21.15.96:0/3860983600 shutdown_connections 2024-03-21T06:36:10.847 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.846+0000 7f54b506ee00 1 --2- 172.21.15.96:0/3860983600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b0cb50df0 0x564b0cb39d30 unknown :-1 s=CLOSED pgs=12856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.847 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.846+0000 7f54b506ee00 1 -- 172.21.15.96:0/3860983600 >> 172.21.15.96:0/3860983600 conn(0x564b0c9cbe00 msgr2=0x564b0cac25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:10.847 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.846+0000 7f54b506ee00 1 -- 172.21.15.96:0/3860983600 shutdown_connections 2024-03-21T06:36:10.847 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.846+0000 7f54b506ee00 1 -- 172.21.15.96:0/3860983600 wait complete. 2024-03-21T06:36:10.847 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.847+0000 7f54b506ee00 1 Processor -- start 2024-03-21T06:36:10.847 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.847+0000 7f54b506ee00 1 -- start start 2024-03-21T06:36:10.850 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.847+0000 7f54b506ee00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b0cb50df0 0x564b0cb3ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:10.850 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.847+0000 7f54b506ee00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b0c9f7550 con 0x564b0cb50df0 2024-03-21T06:36:10.850 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.847+0000 7f54ad59a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b0cb50df0 0x564b0cb3ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:10.850 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.848+0000 7f54ad59a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b0cb50df0 0x564b0cb3ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33232/0 (socket says 172.21.15.96:33232) 2024-03-21T06:36:10.850 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.848+0000 7f54ad59a640 1 -- 172.21.15.96:0/2148504625 learned_addr learned my addr 172.21.15.96:0/2148504625 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:10.850 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.848+0000 7f54ad59a640 1 -- 172.21.15.96:0/2148504625 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b0c9ec680 con 0x564b0cb50df0 2024-03-21T06:36:10.850 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.848+0000 7f54ad59a640 1 --2- 172.21.15.96:0/2148504625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b0cb50df0 0x564b0cb3ccd0 crc :-1 s=READY pgs=12857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:10.850 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.848+0000 7f549e7fc640 1 -- 172.21.15.96:0/2148504625 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f54a4016040 con 0x564b0cb50df0 2024-03-21T06:36:10.851 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.848+0000 7f549e7fc640 1 -- 172.21.15.96:0/2148504625 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f54a4011b00 con 0x564b0cb50df0 2024-03-21T06:36:10.851 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.848+0000 7f54b506ee00 1 -- 172.21.15.96:0/2148504625 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564b0c9ec0e0 con 0x564b0cb50df0 2024-03-21T06:36:10.851 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.848+0000 7f549e7fc640 1 -- 172.21.15.96:0/2148504625 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f54a4011da0 con 0x564b0cb50df0 2024-03-21T06:36:10.851 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.848+0000 7f54b506ee00 1 -- 172.21.15.96:0/2148504625 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b0cab9550 con 0x564b0cb50df0 2024-03-21T06:36:10.851 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.849+0000 7f549e7fc640 1 -- 172.21.15.96:0/2148504625 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f54a400e440 con 0x564b0cb50df0 2024-03-21T06:36:10.851 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.849+0000 7f549e7fc640 1 --2- 172.21.15.96:0/2148504625 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f548c036b80 0x7f548c039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:10.851 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.849+0000 7f549e7fc640 1 -- 172.21.15.96:0/2148504625 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(686..686 src has 402..686) v4 ==== 3791+0+0 (crc 0 0 0) 0x7f54a4048380 con 0x564b0cb50df0 2024-03-21T06:36:10.851 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.849+0000 7f54b506ee00 1 --2- 172.21.15.96:0/2148504625 >> v2:172.21.15.96:6805/13988474 conn(0x564b0cb53d40 0x564b0cb3d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:10.851 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.849+0000 7f54b506ee00 1 -- 172.21.15.96:0/2148504625 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:1 200.1 200:8dd16f86:::obj3:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) v9 -- 0x564b0cfe9f60 con 0x564b0cb53d40 2024-03-21T06:36:10.851 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.849+0000 7f54acd99640 1 --2- 172.21.15.96:0/2148504625 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f548c036b80 0x7f548c039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:10.851 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.850+0000 7f54acd99640 1 --2- 172.21.15.96:0/2148504625 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f548c036b80 0x7f548c039090 crc :-1 s=READY pgs=3828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:10.851 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.850+0000 7f54add9b640 1 --2- 172.21.15.96:0/2148504625 >> v2:172.21.15.96:6805/13988474 conn(0x564b0cb53d40 0x564b0cb3d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:36:10.852 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.852+0000 7f54add9b640 1 --2- 172.21.15.96:0/2148504625 >> v2:172.21.15.96:6805/13988474 conn(0x564b0cb53d40 0x564b0cb3d720 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:10.942 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.942+0000 7f54add9b640 1 -- 172.21.15.96:0/2148504625 <== osd.1 v2:172.21.15.96:6805/13988474 1 ==== osd_op_reply(1 obj3 [writefull 0~11812] v686'1 uv1 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f54a000a430 con 0x564b0cb53d40 2024-03-21T06:36:10.942 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.942+0000 7f54b506ee00 1 -- 172.21.15.96:0/2148504625 >> v2:172.21.15.96:6805/13988474 conn(0x564b0cb53d40 msgr2=0x564b0cb3d720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.942 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.942+0000 7f54b506ee00 1 --2- 172.21.15.96:0/2148504625 >> v2:172.21.15.96:6805/13988474 conn(0x564b0cb53d40 0x564b0cb3d720 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.942 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.942+0000 7f54b506ee00 1 -- 172.21.15.96:0/2148504625 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f548c036b80 msgr2=0x7f548c039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.942 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.942+0000 7f54b506ee00 1 --2- 172.21.15.96:0/2148504625 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f548c036b80 0x7f548c039090 crc :-1 s=READY pgs=3828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.943 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.942+0000 7f54b506ee00 1 -- 172.21.15.96:0/2148504625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b0cb50df0 msgr2=0x564b0cb3ccd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.943 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.942+0000 7f54b506ee00 1 --2- 172.21.15.96:0/2148504625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b0cb50df0 0x564b0cb3ccd0 crc :-1 s=READY pgs=12857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.943 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.942+0000 7f54b506ee00 1 -- 172.21.15.96:0/2148504625 shutdown_connections 2024-03-21T06:36:10.943 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.942+0000 7f54b506ee00 1 --2- 172.21.15.96:0/2148504625 >> v2:172.21.15.96:6805/13988474 conn(0x564b0cb53d40 0x564b0cb3d720 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.943 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.942+0000 7f54b506ee00 1 --2- 172.21.15.96:0/2148504625 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f548c036b80 0x7f548c039090 unknown :-1 s=CLOSED pgs=3828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.943 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.942+0000 7f54b506ee00 1 --2- 172.21.15.96:0/2148504625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b0cb50df0 0x564b0cb3ccd0 unknown :-1 s=CLOSED pgs=12857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.943 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.942+0000 7f54b506ee00 1 -- 172.21.15.96:0/2148504625 >> 172.21.15.96:0/2148504625 conn(0x564b0c9cbe00 msgr2=0x564b0cb39850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:10.943 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.942+0000 7f54b506ee00 1 -- 172.21.15.96:0/2148504625 shutdown_connections 2024-03-21T06:36:10.943 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.942+0000 7f54b506ee00 1 -- 172.21.15.96:0/2148504625 wait complete. 2024-03-21T06:36:10.944 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:36:10.944 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 13dbd6ab-5566-4934-957c-dcb5f3ce8145 put obj4 /etc/passwd 2024-03-21T06:36:10.972 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.973+0000 7f2620677e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:10.976 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.977+0000 7f2620677e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:10.977 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.977+0000 7f2620677e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:10.982 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.978+0000 7f2620677e00 1 Processor -- start 2024-03-21T06:36:10.982 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.978+0000 7f2620677e00 1 -- start start 2024-03-21T06:36:10.982 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.978+0000 7f2620677e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbccfa3040 0x55fbccfa1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:10.982 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.978+0000 7f2620677e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbccf3fea0 con 0x55fbccfa3040 2024-03-21T06:36:10.983 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.978+0000 7f2618b9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbccfa3040 0x55fbccfa1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:10.983 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.978+0000 7f2618b9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbccfa3040 0x55fbccfa1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33244/0 (socket says 172.21.15.96:33244) 2024-03-21T06:36:10.983 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.978+0000 7f2618b9a640 1 -- 172.21.15.96:0/1528453017 learned_addr learned my addr 172.21.15.96:0/1528453017 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:10.983 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.979+0000 7f2618b9a640 1 -- 172.21.15.96:0/1528453017 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbccee2a30 con 0x55fbccfa3040 2024-03-21T06:36:10.983 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.979+0000 7f2618b9a640 1 --2- 172.21.15.96:0/1528453017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbccfa3040 0x55fbccfa1900 crc :-1 s=READY pgs=12860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3591b33b0b05f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:10.983 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.979+0000 7f2612ffd640 1 -- 172.21.15.96:0/1528453017 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2608033040 con 0x55fbccfa3040 2024-03-21T06:36:10.983 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.979+0000 7f2612ffd640 1 -- 172.21.15.96:0/1528453017 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f2608034040 con 0x55fbccfa3040 2024-03-21T06:36:10.983 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.979+0000 7f2612ffd640 1 -- 172.21.15.96:0/1528453017 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f260802ecb0 con 0x55fbccfa3040 2024-03-21T06:36:10.983 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.979+0000 7f2620677e00 1 -- 172.21.15.96:0/1528453017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbccfa3040 msgr2=0x55fbccfa1900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.983 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.979+0000 7f2620677e00 1 --2- 172.21.15.96:0/1528453017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbccfa3040 0x55fbccfa1900 crc :-1 s=READY pgs=12860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.983 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.979+0000 7f2620677e00 1 -- 172.21.15.96:0/1528453017 shutdown_connections 2024-03-21T06:36:10.983 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.979+0000 7f2620677e00 1 --2- 172.21.15.96:0/1528453017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbccfa3040 0x55fbccfa1900 unknown :-1 s=CLOSED pgs=12860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.979+0000 7f2620677e00 1 -- 172.21.15.96:0/1528453017 >> 172.21.15.96:0/1528453017 conn(0x55fbcceb4e00 msgr2=0x55fbcd03f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:10.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.979+0000 7f2620677e00 1 -- 172.21.15.96:0/1528453017 shutdown_connections 2024-03-21T06:36:10.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.979+0000 7f2620677e00 1 -- 172.21.15.96:0/1528453017 wait complete. 2024-03-21T06:36:10.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.980+0000 7f2620677e00 1 Processor -- start 2024-03-21T06:36:10.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.980+0000 7f2620677e00 1 -- start start 2024-03-21T06:36:10.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.981+0000 7f2620677e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbccfa3040 0x55fbcd0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:10.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.981+0000 7f2620677e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fbccee0550 con 0x55fbccfa3040 2024-03-21T06:36:10.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.981+0000 7f2618b9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbccfa3040 0x55fbcd0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:10.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.981+0000 7f2618b9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbccfa3040 0x55fbcd0cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33246/0 (socket says 172.21.15.96:33246) 2024-03-21T06:36:10.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.981+0000 7f2618b9a640 1 -- 172.21.15.96:0/2225170526 learned_addr learned my addr 172.21.15.96:0/2225170526 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:10.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.981+0000 7f2618b9a640 1 -- 172.21.15.96:0/2225170526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fbcced5680 con 0x55fbccfa3040 2024-03-21T06:36:10.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.981+0000 7f2618b9a640 1 --2- 172.21.15.96:0/2225170526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbccfa3040 0x55fbcd0cf070 crc :-1 s=READY pgs=12861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:10.985 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.981+0000 7f2611ffb640 1 -- 172.21.15.96:0/2225170526 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2608033040 con 0x55fbccfa3040 2024-03-21T06:36:10.985 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.981+0000 7f2611ffb640 1 -- 172.21.15.96:0/2225170526 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f2608009540 con 0x55fbccfa3040 2024-03-21T06:36:10.985 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.981+0000 7f2620677e00 1 -- 172.21.15.96:0/2225170526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fbcced50e0 con 0x55fbccfa3040 2024-03-21T06:36:10.985 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.981+0000 7f2611ffb640 1 -- 172.21.15.96:0/2225170526 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f26080097e0 con 0x55fbccfa3040 2024-03-21T06:36:10.985 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.981+0000 7f2620677e00 1 -- 172.21.15.96:0/2225170526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fbcd022fd0 con 0x55fbccfa3040 2024-03-21T06:36:10.985 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.982+0000 7f2611ffb640 1 -- 172.21.15.96:0/2225170526 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f2608009990 con 0x55fbccfa3040 2024-03-21T06:36:10.985 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.982+0000 7f2611ffb640 1 --2- 172.21.15.96:0/2225170526 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f25f8036b80 0x7f25f8039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:10.985 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.982+0000 7f2611ffb640 1 -- 172.21.15.96:0/2225170526 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(686..686 src has 402..686) v4 ==== 3791+0+0 (crc 0 0 0) 0x7f260806b3a0 con 0x55fbccfa3040 2024-03-21T06:36:10.985 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.982+0000 7f2620677e00 1 --2- 172.21.15.96:0/2225170526 >> v2:172.21.15.184:6811/276480837 conn(0x55fbcd038ef0 0x55fbcd03c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:10.985 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.982+0000 7f2620677e00 1 -- 172.21.15.96:0/2225170526 --> v2:172.21.15.184:6811/276480837 -- osd_op(unknown.0.0:1 200.0 200:0ee9ae15:::obj4:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) v9 -- 0x55fbcd023a80 con 0x55fbcd038ef0 2024-03-21T06:36:10.985 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.983+0000 7f2613fff640 1 --2- 172.21.15.96:0/2225170526 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f25f8036b80 0x7f25f8039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:10.985 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.983+0000 7f2613fff640 1 --2- 172.21.15.96:0/2225170526 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f25f8036b80 0x7f25f8039090 crc :-1 s=READY pgs=3829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:10.986 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.983+0000 7f261939b640 1 --2- 172.21.15.96:0/2225170526 >> v2:172.21.15.184:6811/276480837 conn(0x55fbcd038ef0 0x55fbcd03c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:36:10.986 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.985+0000 7f261939b640 1 --2- 172.21.15.96:0/2225170526 >> v2:172.21.15.184:6811/276480837 conn(0x55fbcd038ef0 0x55fbcd03c800 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:10.991 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.991+0000 7f261939b640 1 -- 172.21.15.96:0/2225170526 <== osd.3 v2:172.21.15.184:6811/276480837 1 ==== osd_op_reply(1 obj4 [writefull 0~11812] v686'1 uv1 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f260c00a430 con 0x55fbcd038ef0 2024-03-21T06:36:10.992 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.991+0000 7f2620677e00 1 -- 172.21.15.96:0/2225170526 >> v2:172.21.15.184:6811/276480837 conn(0x55fbcd038ef0 msgr2=0x55fbcd03c800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.992 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.991+0000 7f2620677e00 1 --2- 172.21.15.96:0/2225170526 >> v2:172.21.15.184:6811/276480837 conn(0x55fbcd038ef0 0x55fbcd03c800 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.992 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.991+0000 7f2620677e00 1 -- 172.21.15.96:0/2225170526 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f25f8036b80 msgr2=0x7f25f8039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.992 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.991+0000 7f2620677e00 1 --2- 172.21.15.96:0/2225170526 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f25f8036b80 0x7f25f8039090 crc :-1 s=READY pgs=3829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.992 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.991+0000 7f2620677e00 1 -- 172.21.15.96:0/2225170526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbccfa3040 msgr2=0x55fbcd0cf070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:10.992 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.991+0000 7f2620677e00 1 --2- 172.21.15.96:0/2225170526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbccfa3040 0x55fbcd0cf070 crc :-1 s=READY pgs=12861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.992 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.992+0000 7f2620677e00 1 -- 172.21.15.96:0/2225170526 shutdown_connections 2024-03-21T06:36:10.992 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.992+0000 7f2620677e00 1 --2- 172.21.15.96:0/2225170526 >> v2:172.21.15.184:6811/276480837 conn(0x55fbcd038ef0 0x55fbcd03c800 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.992 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.992+0000 7f2620677e00 1 --2- 172.21.15.96:0/2225170526 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f25f8036b80 0x7f25f8039090 unknown :-1 s=CLOSED pgs=3829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.992 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.992+0000 7f2620677e00 1 --2- 172.21.15.96:0/2225170526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbccfa3040 0x55fbcd0cf070 unknown :-1 s=CLOSED pgs=12861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:10.992 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.992+0000 7f2620677e00 1 -- 172.21.15.96:0/2225170526 >> 172.21.15.96:0/2225170526 conn(0x55fbcceb4e00 msgr2=0x55fbcd03f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:10.992 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.992+0000 7f2620677e00 1 -- 172.21.15.96:0/2225170526 shutdown_connections 2024-03-21T06:36:10.993 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:10.992+0000 7f2620677e00 1 -- 172.21.15.96:0/2225170526 wait complete. 2024-03-21T06:36:10.993 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:36:10.993 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 13dbd6ab-5566-4934-957c-dcb5f3ce8145 put obj5 /etc/passwd 2024-03-21T06:36:11.022 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.023+0000 7f53f923ae00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.026 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.027+0000 7f53f923ae00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.026 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.027+0000 7f53f923ae00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.026 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.027+0000 7f53f923ae00 1 Processor -- start 2024-03-21T06:36:11.026 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.027+0000 7f53f923ae00 1 -- start start 2024-03-21T06:36:11.028 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.028+0000 7f53f923ae00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bafe43570 0x557bafeca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.028 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.028+0000 7f53f923ae00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bafde4ea0 con 0x557bafe43570 2024-03-21T06:36:11.028 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.028+0000 7f53f2249640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bafe43570 0x557bafeca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.028 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.028+0000 7f53f2249640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bafe43570 0x557bafeca020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33264/0 (socket says 172.21.15.96:33264) 2024-03-21T06:36:11.028 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.028+0000 7f53f2249640 1 -- 172.21.15.96:0/1701425550 learned_addr learned my addr 172.21.15.96:0/1701425550 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:11.028 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.028+0000 7f53f2249640 1 -- 172.21.15.96:0/1701425550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bafd87a30 con 0x557bafe43570 2024-03-21T06:36:11.028 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.028+0000 7f53f2249640 1 --2- 172.21.15.96:0/1701425550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bafe43570 0x557bafeca020 crc :-1 s=READY pgs=12864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5a7b33c935a5fb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.028 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.028+0000 7f53f0a46640 1 -- 172.21.15.96:0/1701425550 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f53e800b500 con 0x557bafe43570 2024-03-21T06:36:11.029 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.028+0000 7f53f0a46640 1 -- 172.21.15.96:0/1701425550 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f53e800b660 con 0x557bafe43570 2024-03-21T06:36:11.029 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.028+0000 7f53f0a46640 1 -- 172.21.15.96:0/1701425550 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f53e800b900 con 0x557bafe43570 2024-03-21T06:36:11.032 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.029+0000 7f53f923ae00 1 -- 172.21.15.96:0/1701425550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bafe43570 msgr2=0x557bafeca020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.032 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.029+0000 7f53f923ae00 1 --2- 172.21.15.96:0/1701425550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bafe43570 0x557bafeca020 crc :-1 s=READY pgs=12864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.032 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.029+0000 7f53f923ae00 1 -- 172.21.15.96:0/1701425550 shutdown_connections 2024-03-21T06:36:11.032 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.029+0000 7f53f923ae00 1 --2- 172.21.15.96:0/1701425550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bafe43570 0x557bafeca020 unknown :-1 s=CLOSED pgs=12864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.032 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.029+0000 7f53f923ae00 1 -- 172.21.15.96:0/1701425550 >> 172.21.15.96:0/1701425550 conn(0x557bafd59e00 msgr2=0x557bafe46da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:11.033 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.029+0000 7f53f923ae00 1 -- 172.21.15.96:0/1701425550 shutdown_connections 2024-03-21T06:36:11.033 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.029+0000 7f53f923ae00 1 -- 172.21.15.96:0/1701425550 wait complete. 2024-03-21T06:36:11.033 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.030+0000 7f53f923ae00 1 Processor -- start 2024-03-21T06:36:11.033 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.030+0000 7f53f923ae00 1 -- start start 2024-03-21T06:36:11.033 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.031+0000 7f53f923ae00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bafe43570 0x557bafecd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.033 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.031+0000 7f53f923ae00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bafd85550 con 0x557bafe43570 2024-03-21T06:36:11.033 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.031+0000 7f53f2249640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bafe43570 0x557bafecd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.033 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.031+0000 7f53f2249640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bafe43570 0x557bafecd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33276/0 (socket says 172.21.15.96:33276) 2024-03-21T06:36:11.033 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.031+0000 7f53f2249640 1 -- 172.21.15.96:0/3453815610 learned_addr learned my addr 172.21.15.96:0/3453815610 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:11.033 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.031+0000 7f53f2249640 1 -- 172.21.15.96:0/3453815610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bafd7a680 con 0x557bafe43570 2024-03-21T06:36:11.033 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.031+0000 7f53f2249640 1 --2- 172.21.15.96:0/3453815610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bafe43570 0x557bafecd620 crc :-1 s=READY pgs=12865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.033 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.031+0000 7f53e37fe640 1 -- 172.21.15.96:0/3453815610 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f53e80111e0 con 0x557bafe43570 2024-03-21T06:36:11.033 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.031+0000 7f53e37fe640 1 -- 172.21.15.96:0/3453815610 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f53e80117a0 con 0x557bafe43570 2024-03-21T06:36:11.033 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.031+0000 7f53f923ae00 1 -- 172.21.15.96:0/3453815610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557bafd7a0e0 con 0x557bafe43570 2024-03-21T06:36:11.033 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.031+0000 7f53e37fe640 1 -- 172.21.15.96:0/3453815610 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f53e8011910 con 0x557bafe43570 2024-03-21T06:36:11.034 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.031+0000 7f53f923ae00 1 -- 172.21.15.96:0/3453815610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bafecf030 con 0x557bafe43570 2024-03-21T06:36:11.034 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.032+0000 7f53e37fe640 1 -- 172.21.15.96:0/3453815610 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f53e8011ac0 con 0x557bafe43570 2024-03-21T06:36:11.034 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.032+0000 7f53e37fe640 1 --2- 172.21.15.96:0/3453815610 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f53d0036b80 0x7f53d0039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.034 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.032+0000 7f53e37fe640 1 -- 172.21.15.96:0/3453815610 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(686..686 src has 402..686) v4 ==== 3791+0+0 (crc 0 0 0) 0x7f53e8048250 con 0x557bafe43570 2024-03-21T06:36:11.034 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.032+0000 7f53f923ae00 1 --2- 172.21.15.96:0/3453815610 >> v2:172.21.15.184:6811/276480837 conn(0x557bafe4c590 0x557baff73c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.034 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.032+0000 7f53f923ae00 1 -- 172.21.15.96:0/3453815610 --> v2:172.21.15.184:6811/276480837 -- osd_op(unknown.0.0:1 200.3 200:c52c9666:::obj5:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) v9 -- 0x557bafed10b0 con 0x557bafe4c590 2024-03-21T06:36:11.034 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.033+0000 7f53f1a48640 1 --2- 172.21.15.96:0/3453815610 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f53d0036b80 0x7f53d0039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.034 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.033+0000 7f53f2a4a640 1 --2- 172.21.15.96:0/3453815610 >> v2:172.21.15.184:6811/276480837 conn(0x557bafe4c590 0x557baff73c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:36:11.034 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.033+0000 7f53f1a48640 1 --2- 172.21.15.96:0/3453815610 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f53d0036b80 0x7f53d0039090 crc :-1 s=READY pgs=3830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.035 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.035+0000 7f53f2a4a640 1 --2- 172.21.15.96:0/3453815610 >> v2:172.21.15.184:6811/276480837 conn(0x557bafe4c590 0x557baff73c60 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.042 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.041+0000 7f53f2a4a640 1 -- 172.21.15.96:0/3453815610 <== osd.3 v2:172.21.15.184:6811/276480837 1 ==== osd_op_reply(1 obj5 [writefull 0~11812] v686'1 uv1 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f53e400a430 con 0x557bafe4c590 2024-03-21T06:36:11.042 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.042+0000 7f53f923ae00 1 -- 172.21.15.96:0/3453815610 >> v2:172.21.15.184:6811/276480837 conn(0x557bafe4c590 msgr2=0x557baff73c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.042 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.042+0000 7f53f923ae00 1 --2- 172.21.15.96:0/3453815610 >> v2:172.21.15.184:6811/276480837 conn(0x557bafe4c590 0x557baff73c60 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.042 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.042+0000 7f53f923ae00 1 -- 172.21.15.96:0/3453815610 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f53d0036b80 msgr2=0x7f53d0039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.042 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.042+0000 7f53f923ae00 1 --2- 172.21.15.96:0/3453815610 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f53d0036b80 0x7f53d0039090 crc :-1 s=READY pgs=3830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.042 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.042+0000 7f53f923ae00 1 -- 172.21.15.96:0/3453815610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bafe43570 msgr2=0x557bafecd620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.042 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.042+0000 7f53f923ae00 1 --2- 172.21.15.96:0/3453815610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bafe43570 0x557bafecd620 crc :-1 s=READY pgs=12865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.043 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.042+0000 7f53f923ae00 1 -- 172.21.15.96:0/3453815610 shutdown_connections 2024-03-21T06:36:11.043 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.042+0000 7f53f923ae00 1 --2- 172.21.15.96:0/3453815610 >> v2:172.21.15.184:6811/276480837 conn(0x557bafe4c590 0x557baff73c60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.043 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.042+0000 7f53f923ae00 1 --2- 172.21.15.96:0/3453815610 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f53d0036b80 0x7f53d0039090 unknown :-1 s=CLOSED pgs=3830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.043 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.042+0000 7f53f923ae00 1 --2- 172.21.15.96:0/3453815610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557bafe43570 0x557bafecd620 unknown :-1 s=CLOSED pgs=12865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.043 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.042+0000 7f53f923ae00 1 -- 172.21.15.96:0/3453815610 >> 172.21.15.96:0/3453815610 conn(0x557bafd59e00 msgr2=0x557bafe46d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:11.043 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.042+0000 7f53f923ae00 1 -- 172.21.15.96:0/3453815610 shutdown_connections 2024-03-21T06:36:11.043 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.042+0000 7f53f923ae00 1 -- 172.21.15.96:0/3453815610 wait complete. 2024-03-21T06:36:11.043 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:36:11.043 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 13dbd6ab-5566-4934-957c-dcb5f3ce8145 put obj6 /etc/passwd 2024-03-21T06:36:11.072 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.073+0000 7f3f84a36e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.077 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.077+0000 7f3f84a36e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.077 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.078+0000 7f3f84a36e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.077 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.078+0000 7f3f84a36e00 1 Processor -- start 2024-03-21T06:36:11.077 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.078+0000 7f3f84a36e00 1 -- start start 2024-03-21T06:36:11.079 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.079+0000 7f3f84a36e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d86cfd8570 0x55d86d073410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.080 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.079+0000 7f3f84a36e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86cf79ea0 con 0x55d86cfd8570 2024-03-21T06:36:11.080 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.079+0000 7f3f7db9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d86cfd8570 0x55d86d073410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.080 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.079+0000 7f3f7db9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d86cfd8570 0x55d86d073410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33284/0 (socket says 172.21.15.96:33284) 2024-03-21T06:36:11.080 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.079+0000 7f3f7db9a640 1 -- 172.21.15.96:0/1591422115 learned_addr learned my addr 172.21.15.96:0/1591422115 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:11.080 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.079+0000 7f3f7db9a640 1 -- 172.21.15.96:0/1591422115 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d86cf1ca30 con 0x55d86cfd8570 2024-03-21T06:36:11.080 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.079+0000 7f3f7db9a640 1 --2- 172.21.15.96:0/1591422115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d86cfd8570 0x55d86d073410 crc :-1 s=READY pgs=12868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205b9fcfbd4404f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.080 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.079+0000 7f3f6ffff640 1 -- 172.21.15.96:0/1591422115 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3f7400c490 con 0x55d86cfd8570 2024-03-21T06:36:11.080 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.079+0000 7f3f6ffff640 1 -- 172.21.15.96:0/1591422115 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3f7400c5f0 con 0x55d86cfd8570 2024-03-21T06:36:11.080 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.079+0000 7f3f6ffff640 1 -- 172.21.15.96:0/1591422115 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3f7400c890 con 0x55d86cfd8570 2024-03-21T06:36:11.080 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.080+0000 7f3f84a36e00 1 -- 172.21.15.96:0/1591422115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d86cfd8570 msgr2=0x55d86d073410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.080 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.080+0000 7f3f84a36e00 1 --2- 172.21.15.96:0/1591422115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d86cfd8570 0x55d86d073410 crc :-1 s=READY pgs=12868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.080 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.080+0000 7f3f84a36e00 1 -- 172.21.15.96:0/1591422115 shutdown_connections 2024-03-21T06:36:11.080 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.080+0000 7f3f84a36e00 1 --2- 172.21.15.96:0/1591422115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d86cfd8570 0x55d86d073410 unknown :-1 s=CLOSED pgs=12868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.080 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.080+0000 7f3f84a36e00 1 -- 172.21.15.96:0/1591422115 >> 172.21.15.96:0/1591422115 conn(0x55d86ceeee00 msgr2=0x55d86cfdbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:11.081 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.080+0000 7f3f84a36e00 1 -- 172.21.15.96:0/1591422115 shutdown_connections 2024-03-21T06:36:11.081 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.080+0000 7f3f84a36e00 1 -- 172.21.15.96:0/1591422115 wait complete. 2024-03-21T06:36:11.081 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.081+0000 7f3f84a36e00 1 Processor -- start 2024-03-21T06:36:11.081 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.081+0000 7f3f84a36e00 1 -- start start 2024-03-21T06:36:11.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.081+0000 7f3f84a36e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d86cfd8570 0x55d86d0654e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.081+0000 7f3f84a36e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86cf1a550 con 0x55d86cfd8570 2024-03-21T06:36:11.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.081+0000 7f3f7db9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d86cfd8570 0x55d86d0654e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.081+0000 7f3f7db9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d86cfd8570 0x55d86d0654e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33288/0 (socket says 172.21.15.96:33288) 2024-03-21T06:36:11.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.081+0000 7f3f7db9a640 1 -- 172.21.15.96:0/948469616 learned_addr learned my addr 172.21.15.96:0/948469616 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:11.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.082+0000 7f3f7db9a640 1 -- 172.21.15.96:0/948469616 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d86cf0f680 con 0x55d86cfd8570 2024-03-21T06:36:11.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.082+0000 7f3f7db9a640 1 --2- 172.21.15.96:0/948469616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d86cfd8570 0x55d86d0654e0 crc :-1 s=READY pgs=12869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.082+0000 7f3f6effd640 1 -- 172.21.15.96:0/948469616 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3f740111e0 con 0x55d86cfd8570 2024-03-21T06:36:11.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.082+0000 7f3f6effd640 1 -- 172.21.15.96:0/948469616 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3f740119d0 con 0x55d86cfd8570 2024-03-21T06:36:11.084 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.082+0000 7f3f6effd640 1 -- 172.21.15.96:0/948469616 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3f74011b40 con 0x55d86cfd8570 2024-03-21T06:36:11.084 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.082+0000 7f3f84a36e00 1 -- 172.21.15.96:0/948469616 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d86cf0f0e0 con 0x55d86cfd8570 2024-03-21T06:36:11.084 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.082+0000 7f3f84a36e00 1 -- 172.21.15.96:0/948469616 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d86d068930 con 0x55d86cfd8570 2024-03-21T06:36:11.084 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.083+0000 7f3f6effd640 1 -- 172.21.15.96:0/948469616 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f3f74011cf0 con 0x55d86cfd8570 2024-03-21T06:36:11.084 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.083+0000 7f3f6effd640 1 --2- 172.21.15.96:0/948469616 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3f5c036b80 0x7f3f5c039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.084 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.083+0000 7f3f6effd640 1 -- 172.21.15.96:0/948469616 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(686..686 src has 402..686) v4 ==== 3791+0+0 (crc 0 0 0) 0x7f3f74048290 con 0x55d86cfd8570 2024-03-21T06:36:11.085 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.083+0000 7f3f84a36e00 1 --2- 172.21.15.96:0/948469616 >> v2:172.21.15.96:6805/13988474 conn(0x55d86cfe1590 0x55d86d10ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.085 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.083+0000 7f3f84a36e00 1 -- 172.21.15.96:0/948469616 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:1 200.4 200:233a42c1:::obj6:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) v9 -- 0x55d86d068fd0 con 0x55d86cfe1590 2024-03-21T06:36:11.085 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.083+0000 7f3f7d399640 1 --2- 172.21.15.96:0/948469616 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3f5c036b80 0x7f3f5c039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.085 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.084+0000 7f3f7e39b640 1 --2- 172.21.15.96:0/948469616 >> v2:172.21.15.96:6805/13988474 conn(0x55d86cfe1590 0x55d86d10ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:36:11.085 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.084+0000 7f3f7d399640 1 --2- 172.21.15.96:0/948469616 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3f5c036b80 0x7f3f5c039090 crc :-1 s=READY pgs=3831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.085 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.085+0000 7f3f7e39b640 1 --2- 172.21.15.96:0/948469616 >> v2:172.21.15.96:6805/13988474 conn(0x55d86cfe1590 0x55d86d10ac80 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.092 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.092+0000 7f3f7e39b640 1 -- 172.21.15.96:0/948469616 <== osd.1 v2:172.21.15.96:6805/13988474 1 ==== osd_op_reply(1 obj6 [writefull 0~11812] v686'1 uv1 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f3f7000a430 con 0x55d86cfe1590 2024-03-21T06:36:11.092 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.092+0000 7f3f84a36e00 1 -- 172.21.15.96:0/948469616 >> v2:172.21.15.96:6805/13988474 conn(0x55d86cfe1590 msgr2=0x55d86d10ac80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.092 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.092+0000 7f3f84a36e00 1 --2- 172.21.15.96:0/948469616 >> v2:172.21.15.96:6805/13988474 conn(0x55d86cfe1590 0x55d86d10ac80 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.092 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.092+0000 7f3f84a36e00 1 -- 172.21.15.96:0/948469616 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3f5c036b80 msgr2=0x7f3f5c039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.093 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.092+0000 7f3f84a36e00 1 --2- 172.21.15.96:0/948469616 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3f5c036b80 0x7f3f5c039090 crc :-1 s=READY pgs=3831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.093 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.092+0000 7f3f84a36e00 1 -- 172.21.15.96:0/948469616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d86cfd8570 msgr2=0x55d86d0654e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.093 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.092+0000 7f3f84a36e00 1 --2- 172.21.15.96:0/948469616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d86cfd8570 0x55d86d0654e0 crc :-1 s=READY pgs=12869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.093 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.092+0000 7f3f84a36e00 1 -- 172.21.15.96:0/948469616 shutdown_connections 2024-03-21T06:36:11.093 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.092+0000 7f3f84a36e00 1 --2- 172.21.15.96:0/948469616 >> v2:172.21.15.96:6805/13988474 conn(0x55d86cfe1590 0x55d86d10ac80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.093 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.092+0000 7f3f84a36e00 1 --2- 172.21.15.96:0/948469616 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3f5c036b80 0x7f3f5c039090 unknown :-1 s=CLOSED pgs=3831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.093 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.092+0000 7f3f84a36e00 1 --2- 172.21.15.96:0/948469616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d86cfd8570 0x55d86d0654e0 unknown :-1 s=CLOSED pgs=12869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.093 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.092+0000 7f3f84a36e00 1 -- 172.21.15.96:0/948469616 >> 172.21.15.96:0/948469616 conn(0x55d86ceeee00 msgr2=0x55d86cfdbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:11.093 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.092+0000 7f3f84a36e00 1 -- 172.21.15.96:0/948469616 shutdown_connections 2024-03-21T06:36:11.093 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.092+0000 7f3f84a36e00 1 -- 172.21.15.96:0/948469616 wait complete. 2024-03-21T06:36:11.094 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:36:11.094 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 13dbd6ab-5566-4934-957c-dcb5f3ce8145 put obj7 /etc/passwd 2024-03-21T06:36:11.123 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.124+0000 7f3b6f279e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.127 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.128+0000 7f3b6f279e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.127 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.128+0000 7f3b6f279e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.128 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.128+0000 7f3b6f279e00 1 Processor -- start 2024-03-21T06:36:11.128 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.129+0000 7f3b6f279e00 1 -- start start 2024-03-21T06:36:11.130 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.129+0000 7f3b6f279e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564878beddf0 0x564878c7a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.130 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.129+0000 7f3b6f279e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x564878af3ea0 con 0x564878beddf0 2024-03-21T06:36:11.130 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.129+0000 7f3b6779a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564878beddf0 0x564878c7a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.130 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.129+0000 7f3b6779a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564878beddf0 0x564878c7a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33308/0 (socket says 172.21.15.96:33308) 2024-03-21T06:36:11.130 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.129+0000 7f3b6779a640 1 -- 172.21.15.96:0/1005806578 learned_addr learned my addr 172.21.15.96:0/1005806578 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:11.130 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.129+0000 7f3b6779a640 1 -- 172.21.15.96:0/1005806578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564878a96a30 con 0x564878beddf0 2024-03-21T06:36:11.130 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.130+0000 7f3b6779a640 1 --2- 172.21.15.96:0/1005806578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564878beddf0 0x564878c7a530 crc :-1 s=READY pgs=12872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5aa631bb3fbc839 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.130 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.130+0000 7f3b65f97640 1 -- 172.21.15.96:0/1005806578 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3b5800c490 con 0x564878beddf0 2024-03-21T06:36:11.131 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.130+0000 7f3b65f97640 1 -- 172.21.15.96:0/1005806578 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3b5800c5f0 con 0x564878beddf0 2024-03-21T06:36:11.131 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.130+0000 7f3b65f97640 1 -- 172.21.15.96:0/1005806578 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3b5800c890 con 0x564878beddf0 2024-03-21T06:36:11.131 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.130+0000 7f3b6f279e00 1 -- 172.21.15.96:0/1005806578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564878beddf0 msgr2=0x564878c7a530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.131 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.130+0000 7f3b6f279e00 1 --2- 172.21.15.96:0/1005806578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564878beddf0 0x564878c7a530 crc :-1 s=READY pgs=12872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.131 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.130+0000 7f3b6f279e00 1 -- 172.21.15.96:0/1005806578 shutdown_connections 2024-03-21T06:36:11.131 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.130+0000 7f3b6f279e00 1 --2- 172.21.15.96:0/1005806578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564878beddf0 0x564878c7a530 unknown :-1 s=CLOSED pgs=12872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.131 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.130+0000 7f3b6f279e00 1 -- 172.21.15.96:0/1005806578 >> 172.21.15.96:0/1005806578 conn(0x564878a68e00 msgr2=0x564878b55300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:11.131 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.130+0000 7f3b6f279e00 1 -- 172.21.15.96:0/1005806578 shutdown_connections 2024-03-21T06:36:11.131 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.130+0000 7f3b6f279e00 1 -- 172.21.15.96:0/1005806578 wait complete. 2024-03-21T06:36:11.131 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.131+0000 7f3b6f279e00 1 Processor -- start 2024-03-21T06:36:11.131 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.131+0000 7f3b6f279e00 1 -- start start 2024-03-21T06:36:11.133 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.132+0000 7f3b6f279e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564878beddf0 0x564878c82750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.134 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.132+0000 7f3b6f279e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x564878a94550 con 0x564878beddf0 2024-03-21T06:36:11.134 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.132+0000 7f3b6779a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564878beddf0 0x564878c82750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.134 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.132+0000 7f3b6779a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564878beddf0 0x564878c82750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33324/0 (socket says 172.21.15.96:33324) 2024-03-21T06:36:11.134 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.132+0000 7f3b6779a640 1 -- 172.21.15.96:0/1888756109 learned_addr learned my addr 172.21.15.96:0/1888756109 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:11.134 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.132+0000 7f3b6779a640 1 -- 172.21.15.96:0/1888756109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564878a89680 con 0x564878beddf0 2024-03-21T06:36:11.134 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.132+0000 7f3b6779a640 1 --2- 172.21.15.96:0/1888756109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564878beddf0 0x564878c82750 crc :-1 s=READY pgs=12873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.134 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.132+0000 7f3b64f95640 1 -- 172.21.15.96:0/1888756109 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3b580111e0 con 0x564878beddf0 2024-03-21T06:36:11.134 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.132+0000 7f3b64f95640 1 -- 172.21.15.96:0/1888756109 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3b58011980 con 0x564878beddf0 2024-03-21T06:36:11.134 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.132+0000 7f3b6f279e00 1 -- 172.21.15.96:0/1888756109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564878a890e0 con 0x564878beddf0 2024-03-21T06:36:11.134 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.132+0000 7f3b64f95640 1 -- 172.21.15.96:0/1888756109 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3b58011af0 con 0x564878beddf0 2024-03-21T06:36:11.134 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.132+0000 7f3b6f279e00 1 -- 172.21.15.96:0/1888756109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564878c772f0 con 0x564878beddf0 2024-03-21T06:36:11.134 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.133+0000 7f3b64f95640 1 -- 172.21.15.96:0/1888756109 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f3b58011ca0 con 0x564878beddf0 2024-03-21T06:36:11.135 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.133+0000 7f3b64f95640 1 --2- 172.21.15.96:0/1888756109 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3b4c036b80 0x7f3b4c039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.135 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.133+0000 7f3b64f95640 1 -- 172.21.15.96:0/1888756109 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(686..686 src has 402..686) v4 ==== 3791+0+0 (crc 0 0 0) 0x7f3b580482f0 con 0x564878beddf0 2024-03-21T06:36:11.135 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.133+0000 7f3b6f279e00 1 --2- 172.21.15.96:0/1888756109 >> v2:172.21.15.96:6801/932229279 conn(0x564878bf0d40 0x564878bec520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.135 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.133+0000 7f3b66f99640 1 --2- 172.21.15.96:0/1888756109 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3b4c036b80 0x7f3b4c039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.135 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.133+0000 7f3b6f279e00 1 -- 172.21.15.96:0/1888756109 --> v2:172.21.15.96:6801/932229279 -- osd_op(unknown.0.0:1 200.9 200:9a09113a:::obj7:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) v9 -- 0x564878bd7210 con 0x564878bf0d40 2024-03-21T06:36:11.135 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.134+0000 7f3b66f99640 1 --2- 172.21.15.96:0/1888756109 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3b4c036b80 0x7f3b4c039090 crc :-1 s=READY pgs=3832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.135 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.134+0000 7f3b67f9b640 1 --2- 172.21.15.96:0/1888756109 >> v2:172.21.15.96:6801/932229279 conn(0x564878bf0d40 0x564878bec520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:36:11.135 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.136+0000 7f3b67f9b640 1 --2- 172.21.15.96:0/1888756109 >> v2:172.21.15.96:6801/932229279 conn(0x564878bf0d40 0x564878bec520 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.143 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.142+0000 7f3b67f9b640 1 -- 172.21.15.96:0/1888756109 <== osd.0 v2:172.21.15.96:6801/932229279 1 ==== osd_op_reply(1 obj7 [writefull 0~11812] v686'1 uv1 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f3b5000a430 con 0x564878bf0d40 2024-03-21T06:36:11.143 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.142+0000 7f3b6f279e00 1 -- 172.21.15.96:0/1888756109 >> v2:172.21.15.96:6801/932229279 conn(0x564878bf0d40 msgr2=0x564878bec520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.143 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.142+0000 7f3b6f279e00 1 --2- 172.21.15.96:0/1888756109 >> v2:172.21.15.96:6801/932229279 conn(0x564878bf0d40 0x564878bec520 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.143 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.142+0000 7f3b6f279e00 1 -- 172.21.15.96:0/1888756109 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3b4c036b80 msgr2=0x7f3b4c039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.143 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.142+0000 7f3b6f279e00 1 --2- 172.21.15.96:0/1888756109 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3b4c036b80 0x7f3b4c039090 crc :-1 s=READY pgs=3832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.143 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.143+0000 7f3b6f279e00 1 -- 172.21.15.96:0/1888756109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564878beddf0 msgr2=0x564878c82750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.143 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.143+0000 7f3b6f279e00 1 --2- 172.21.15.96:0/1888756109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564878beddf0 0x564878c82750 crc :-1 s=READY pgs=12873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.143 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.143+0000 7f3b6f279e00 1 -- 172.21.15.96:0/1888756109 shutdown_connections 2024-03-21T06:36:11.143 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.143+0000 7f3b6f279e00 1 --2- 172.21.15.96:0/1888756109 >> v2:172.21.15.96:6801/932229279 conn(0x564878bf0d40 0x564878bec520 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.143 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.143+0000 7f3b6f279e00 1 --2- 172.21.15.96:0/1888756109 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3b4c036b80 0x7f3b4c039090 unknown :-1 s=CLOSED pgs=3832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.144 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.143+0000 7f3b6f279e00 1 --2- 172.21.15.96:0/1888756109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564878beddf0 0x564878c82750 unknown :-1 s=CLOSED pgs=12873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.144 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.143+0000 7f3b6f279e00 1 -- 172.21.15.96:0/1888756109 >> 172.21.15.96:0/1888756109 conn(0x564878a68e00 msgr2=0x564878b55300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:11.144 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.143+0000 7f3b6f279e00 1 -- 172.21.15.96:0/1888756109 shutdown_connections 2024-03-21T06:36:11.144 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.143+0000 7f3b6f279e00 1 -- 172.21.15.96:0/1888756109 wait complete. 2024-03-21T06:36:11.144 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:36:11.144 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 13dbd6ab-5566-4934-957c-dcb5f3ce8145 put obj8 /etc/passwd 2024-03-21T06:36:11.174 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.174+0000 7f69b286ee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.177 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.178+0000 7f69b286ee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.178 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.178+0000 7f69b286ee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.178 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.179+0000 7f69b286ee00 1 Processor -- start 2024-03-21T06:36:11.178 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.179+0000 7f69b286ee00 1 -- start start 2024-03-21T06:36:11.179 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.180+0000 7f69b286ee00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d25c5b7df0 0x55d25c5a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.179 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.180+0000 7f69b286ee00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25c4bdea0 con 0x55d25c5b7df0 2024-03-21T06:36:11.180 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.180+0000 7f69aad9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d25c5b7df0 0x55d25c5a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.180 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.180+0000 7f69aad9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d25c5b7df0 0x55d25c5a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33336/0 (socket says 172.21.15.96:33336) 2024-03-21T06:36:11.180 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.180+0000 7f69aad9a640 1 -- 172.21.15.96:0/715767894 learned_addr learned my addr 172.21.15.96:0/715767894 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:11.180 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.180+0000 7f69aad9a640 1 -- 172.21.15.96:0/715767894 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d25c460a30 con 0x55d25c5b7df0 2024-03-21T06:36:11.180 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.180+0000 7f69aad9a640 1 --2- 172.21.15.96:0/715767894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d25c5b7df0 0x55d25c5a0d30 crc :-1 s=READY pgs=12876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c14c151bd0c3da server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.180 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.180+0000 7f69a9597640 1 -- 172.21.15.96:0/715767894 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f69a000b500 con 0x55d25c5b7df0 2024-03-21T06:36:11.180 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.180+0000 7f69a9597640 1 -- 172.21.15.96:0/715767894 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f69a000b660 con 0x55d25c5b7df0 2024-03-21T06:36:11.180 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.180+0000 7f69a9597640 1 -- 172.21.15.96:0/715767894 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f69a000b900 con 0x55d25c5b7df0 2024-03-21T06:36:11.182 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.181+0000 7f69b286ee00 1 -- 172.21.15.96:0/715767894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d25c5b7df0 msgr2=0x55d25c5a0d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.182 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.181+0000 7f69b286ee00 1 --2- 172.21.15.96:0/715767894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d25c5b7df0 0x55d25c5a0d30 crc :-1 s=READY pgs=12876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.182 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.181+0000 7f69b286ee00 1 -- 172.21.15.96:0/715767894 shutdown_connections 2024-03-21T06:36:11.182 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.181+0000 7f69b286ee00 1 --2- 172.21.15.96:0/715767894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d25c5b7df0 0x55d25c5a0d30 unknown :-1 s=CLOSED pgs=12876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.183 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.181+0000 7f69b286ee00 1 -- 172.21.15.96:0/715767894 >> 172.21.15.96:0/715767894 conn(0x55d25c432e00 msgr2=0x55d25c5295e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:11.183 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.181+0000 7f69b286ee00 1 -- 172.21.15.96:0/715767894 shutdown_connections 2024-03-21T06:36:11.183 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.181+0000 7f69b286ee00 1 -- 172.21.15.96:0/715767894 wait complete. 2024-03-21T06:36:11.183 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.181+0000 7f69b286ee00 1 Processor -- start 2024-03-21T06:36:11.183 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.181+0000 7f69b286ee00 1 -- start start 2024-03-21T06:36:11.183 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.182+0000 7f69b286ee00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d25c5b7df0 0x55d25c5a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.183 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.182+0000 7f69b286ee00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25c45e550 con 0x55d25c5b7df0 2024-03-21T06:36:11.183 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.182+0000 7f69aad9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d25c5b7df0 0x55d25c5a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.183 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.182+0000 7f69aad9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d25c5b7df0 0x55d25c5a3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33338/0 (socket says 172.21.15.96:33338) 2024-03-21T06:36:11.183 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.182+0000 7f69aad9a640 1 -- 172.21.15.96:0/1405027173 learned_addr learned my addr 172.21.15.96:0/1405027173 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:11.183 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.182+0000 7f69aad9a640 1 -- 172.21.15.96:0/1405027173 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d25c453680 con 0x55d25c5b7df0 2024-03-21T06:36:11.183 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.183+0000 7f69aad9a640 1 --2- 172.21.15.96:0/1405027173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d25c5b7df0 0x55d25c5a3cd0 crc :-1 s=READY pgs=12877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.183 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.183+0000 7f6993fff640 1 -- 172.21.15.96:0/1405027173 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f69a00110b0 con 0x55d25c5b7df0 2024-03-21T06:36:11.183 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.183+0000 7f6993fff640 1 -- 172.21.15.96:0/1405027173 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f69a0011210 con 0x55d25c5b7df0 2024-03-21T06:36:11.183 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.183+0000 7f69b286ee00 1 -- 172.21.15.96:0/1405027173 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d25c4530e0 con 0x55d25c5b7df0 2024-03-21T06:36:11.184 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.183+0000 7f6993fff640 1 -- 172.21.15.96:0/1405027173 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f69a0011a60 con 0x55d25c5b7df0 2024-03-21T06:36:11.184 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.183+0000 7f69b286ee00 1 -- 172.21.15.96:0/1405027173 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d25c520550 con 0x55d25c5b7df0 2024-03-21T06:36:11.185 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.184+0000 7f6993fff640 1 -- 172.21.15.96:0/1405027173 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f69a0011c10 con 0x55d25c5b7df0 2024-03-21T06:36:11.185 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.184+0000 7f6993fff640 1 --2- 172.21.15.96:0/1405027173 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f698c036b80 0x7f698c039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.185 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.184+0000 7f6993fff640 1 -- 172.21.15.96:0/1405027173 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(686..686 src has 402..686) v4 ==== 3791+0+0 (crc 0 0 0) 0x7f69a0048150 con 0x55d25c5b7df0 2024-03-21T06:36:11.185 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.184+0000 7f69b286ee00 1 --2- 172.21.15.96:0/1405027173 >> v2:172.21.15.96:6801/932229279 conn(0x55d25c5bad40 0x55d25c5a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.185 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.184+0000 7f69b286ee00 1 -- 172.21.15.96:0/1405027173 --> v2:172.21.15.96:6801/932229279 -- osd_op(unknown.0.0:1 200.5 200:a557efb1:::obj8:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) v9 -- 0x55d25ca50f60 con 0x55d25c5bad40 2024-03-21T06:36:11.185 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.184+0000 7f69aa599640 1 --2- 172.21.15.96:0/1405027173 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f698c036b80 0x7f698c039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.185 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.184+0000 7f69aa599640 1 --2- 172.21.15.96:0/1405027173 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f698c036b80 0x7f698c039090 crc :-1 s=READY pgs=3833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.185 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.184+0000 7f69ab59b640 1 --2- 172.21.15.96:0/1405027173 >> v2:172.21.15.96:6801/932229279 conn(0x55d25c5bad40 0x55d25c5a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:36:11.186 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.186+0000 7f69ab59b640 1 --2- 172.21.15.96:0/1405027173 >> v2:172.21.15.96:6801/932229279 conn(0x55d25c5bad40 0x55d25c5a4720 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.192 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.192+0000 7f69ab59b640 1 -- 172.21.15.96:0/1405027173 <== osd.0 v2:172.21.15.96:6801/932229279 1 ==== osd_op_reply(1 obj8 [writefull 0~11812] v686'1 uv1 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f699c00a430 con 0x55d25c5bad40 2024-03-21T06:36:11.192 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.193+0000 7f69b286ee00 1 -- 172.21.15.96:0/1405027173 >> v2:172.21.15.96:6801/932229279 conn(0x55d25c5bad40 msgr2=0x55d25c5a4720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.193 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.193+0000 7f69b286ee00 1 --2- 172.21.15.96:0/1405027173 >> v2:172.21.15.96:6801/932229279 conn(0x55d25c5bad40 0x55d25c5a4720 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.193 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.193+0000 7f69b286ee00 1 -- 172.21.15.96:0/1405027173 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f698c036b80 msgr2=0x7f698c039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.193 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.193+0000 7f69b286ee00 1 --2- 172.21.15.96:0/1405027173 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f698c036b80 0x7f698c039090 crc :-1 s=READY pgs=3833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.193 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.193+0000 7f69b286ee00 1 -- 172.21.15.96:0/1405027173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d25c5b7df0 msgr2=0x55d25c5a3cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.193 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.193+0000 7f69b286ee00 1 --2- 172.21.15.96:0/1405027173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d25c5b7df0 0x55d25c5a3cd0 crc :-1 s=READY pgs=12877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.193 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.193+0000 7f69b286ee00 1 -- 172.21.15.96:0/1405027173 shutdown_connections 2024-03-21T06:36:11.193 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.193+0000 7f69b286ee00 1 --2- 172.21.15.96:0/1405027173 >> v2:172.21.15.96:6801/932229279 conn(0x55d25c5bad40 0x55d25c5a4720 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.193 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.193+0000 7f69b286ee00 1 --2- 172.21.15.96:0/1405027173 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f698c036b80 0x7f698c039090 unknown :-1 s=CLOSED pgs=3833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.193 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.193+0000 7f69b286ee00 1 --2- 172.21.15.96:0/1405027173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d25c5b7df0 0x55d25c5a3cd0 unknown :-1 s=CLOSED pgs=12877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.193 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.193+0000 7f69b286ee00 1 -- 172.21.15.96:0/1405027173 >> 172.21.15.96:0/1405027173 conn(0x55d25c432e00 msgr2=0x55d25c5a0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:11.193 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.193+0000 7f69b286ee00 1 -- 172.21.15.96:0/1405027173 shutdown_connections 2024-03-21T06:36:11.193 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.193+0000 7f69b286ee00 1 -- 172.21.15.96:0/1405027173 wait complete. 2024-03-21T06:36:11.194 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:36:11.194 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 13dbd6ab-5566-4934-957c-dcb5f3ce8145 put obj9 /etc/passwd 2024-03-21T06:36:11.224 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.225+0000 7fb7dda72e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.228 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.229+0000 7fb7dda72e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.228 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.229+0000 7fb7dda72e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.229 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.230+0000 7fb7dda72e00 1 Processor -- start 2024-03-21T06:36:11.229 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.230+0000 7fb7dda72e00 1 -- start start 2024-03-21T06:36:11.231 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.230+0000 7fb7dda72e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589ad5f5df0 0x5589ad5ded30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.231 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.230+0000 7fb7dda72e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ad4fbea0 con 0x5589ad5f5df0 2024-03-21T06:36:11.231 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.230+0000 7fb7d5f9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589ad5f5df0 0x5589ad5ded30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.231 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.230+0000 7fb7d5f9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589ad5f5df0 0x5589ad5ded30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33356/0 (socket says 172.21.15.96:33356) 2024-03-21T06:36:11.231 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.230+0000 7fb7d5f9a640 1 -- 172.21.15.96:0/2906845093 learned_addr learned my addr 172.21.15.96:0/2906845093 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:11.231 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.230+0000 7fb7d5f9a640 1 -- 172.21.15.96:0/2906845093 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589ad49ea30 con 0x5589ad5f5df0 2024-03-21T06:36:11.231 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.231+0000 7fb7d5f9a640 1 --2- 172.21.15.96:0/2906845093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589ad5f5df0 0x5589ad5ded30 crc :-1 s=READY pgs=12880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e7e7b2ca9bf914 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.231 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.231+0000 7fb7cffff640 1 -- 172.21.15.96:0/2906845093 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb7c00060b0 con 0x5589ad5f5df0 2024-03-21T06:36:11.231 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.231+0000 7fb7cffff640 1 -- 172.21.15.96:0/2906845093 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb7c0006210 con 0x5589ad5f5df0 2024-03-21T06:36:11.232 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.231+0000 7fb7cffff640 1 -- 172.21.15.96:0/2906845093 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb7c0001b10 con 0x5589ad5f5df0 2024-03-21T06:36:11.232 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.231+0000 7fb7dda72e00 1 -- 172.21.15.96:0/2906845093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589ad5f5df0 msgr2=0x5589ad5ded30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.232 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.231+0000 7fb7dda72e00 1 --2- 172.21.15.96:0/2906845093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589ad5f5df0 0x5589ad5ded30 crc :-1 s=READY pgs=12880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.232 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.231+0000 7fb7dda72e00 1 -- 172.21.15.96:0/2906845093 shutdown_connections 2024-03-21T06:36:11.232 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.231+0000 7fb7dda72e00 1 --2- 172.21.15.96:0/2906845093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589ad5f5df0 0x5589ad5ded30 unknown :-1 s=CLOSED pgs=12880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.232 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.231+0000 7fb7dda72e00 1 -- 172.21.15.96:0/2906845093 >> 172.21.15.96:0/2906845093 conn(0x5589ad470e00 msgr2=0x5589ad5675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:11.232 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.231+0000 7fb7dda72e00 1 -- 172.21.15.96:0/2906845093 shutdown_connections 2024-03-21T06:36:11.232 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.231+0000 7fb7dda72e00 1 -- 172.21.15.96:0/2906845093 wait complete. 2024-03-21T06:36:11.232 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.232+0000 7fb7dda72e00 1 Processor -- start 2024-03-21T06:36:11.232 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.232+0000 7fb7dda72e00 1 -- start start 2024-03-21T06:36:11.234 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.232+0000 7fb7dda72e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589ad5f5df0 0x5589ad5e1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.234 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.232+0000 7fb7dda72e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ad49c550 con 0x5589ad5f5df0 2024-03-21T06:36:11.235 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.233+0000 7fb7d5f9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589ad5f5df0 0x5589ad5e1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.235 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.233+0000 7fb7d5f9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589ad5f5df0 0x5589ad5e1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33360/0 (socket says 172.21.15.96:33360) 2024-03-21T06:36:11.235 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.233+0000 7fb7d5f9a640 1 -- 172.21.15.96:0/1719061433 learned_addr learned my addr 172.21.15.96:0/1719061433 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:11.235 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.233+0000 7fb7d5f9a640 1 -- 172.21.15.96:0/1719061433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589ad491680 con 0x5589ad5f5df0 2024-03-21T06:36:11.235 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.233+0000 7fb7d5f9a640 1 --2- 172.21.15.96:0/1719061433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589ad5f5df0 0x5589ad5e1cd0 crc :-1 s=READY pgs=12881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.235 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.233+0000 7fb7ceffd640 1 -- 172.21.15.96:0/1719061433 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb7c00068b0 con 0x5589ad5f5df0 2024-03-21T06:36:11.235 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.233+0000 7fb7ceffd640 1 -- 172.21.15.96:0/1719061433 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb7c0004b50 con 0x5589ad5f5df0 2024-03-21T06:36:11.235 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.233+0000 7fb7dda72e00 1 -- 172.21.15.96:0/1719061433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5589ad4910e0 con 0x5589ad5f5df0 2024-03-21T06:36:11.235 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.233+0000 7fb7ceffd640 1 -- 172.21.15.96:0/1719061433 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb7c0004df0 con 0x5589ad5f5df0 2024-03-21T06:36:11.235 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.233+0000 7fb7dda72e00 1 -- 172.21.15.96:0/1719061433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589ad55e550 con 0x5589ad5f5df0 2024-03-21T06:36:11.235 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.234+0000 7fb7ceffd640 1 -- 172.21.15.96:0/1719061433 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fb7c0013340 con 0x5589ad5f5df0 2024-03-21T06:36:11.235 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.234+0000 7fb7ceffd640 1 --2- 172.21.15.96:0/1719061433 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb7bc036b30 0x7fb7bc039040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.235 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.234+0000 7fb7ceffd640 1 -- 172.21.15.96:0/1719061433 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(686..686 src has 402..686) v4 ==== 3791+0+0 (crc 0 0 0) 0x7fb7c0047670 con 0x5589ad5f5df0 2024-03-21T06:36:11.235 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.234+0000 7fb7dda72e00 1 --2- 172.21.15.96:0/1719061433 >> v2:172.21.15.184:6811/276480837 conn(0x5589ad5f8d40 0x5589ad5e2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.235 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.234+0000 7fb7dda72e00 1 -- 172.21.15.96:0/1719061433 --> v2:172.21.15.184:6811/276480837 -- osd_op(unknown.0.0:1 200.7 200:fe351e27:::obj9:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) v9 -- 0x5589ada8ef60 con 0x5589ad5f8d40 2024-03-21T06:36:11.236 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.234+0000 7fb7d5799640 1 --2- 172.21.15.96:0/1719061433 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb7bc036b30 0x7fb7bc039040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.236 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.235+0000 7fb7d679b640 1 --2- 172.21.15.96:0/1719061433 >> v2:172.21.15.184:6811/276480837 conn(0x5589ad5f8d40 0x5589ad5e2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:36:11.236 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.235+0000 7fb7d5799640 1 --2- 172.21.15.96:0/1719061433 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb7bc036b30 0x7fb7bc039040 crc :-1 s=READY pgs=3834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.236 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.236+0000 7fb7d679b640 1 --2- 172.21.15.96:0/1719061433 >> v2:172.21.15.184:6811/276480837 conn(0x5589ad5f8d40 0x5589ad5e2720 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.242 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.243+0000 7fb7d679b640 1 -- 172.21.15.96:0/1719061433 <== osd.3 v2:172.21.15.184:6811/276480837 1 ==== osd_op_reply(1 obj9 [writefull 0~11812] v686'2 uv2 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb7c800c430 con 0x5589ad5f8d40 2024-03-21T06:36:11.243 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.243+0000 7fb7dda72e00 1 -- 172.21.15.96:0/1719061433 >> v2:172.21.15.184:6811/276480837 conn(0x5589ad5f8d40 msgr2=0x5589ad5e2720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.243 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.243+0000 7fb7dda72e00 1 --2- 172.21.15.96:0/1719061433 >> v2:172.21.15.184:6811/276480837 conn(0x5589ad5f8d40 0x5589ad5e2720 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.243 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.243+0000 7fb7dda72e00 1 -- 172.21.15.96:0/1719061433 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb7bc036b30 msgr2=0x7fb7bc039040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.243 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.243+0000 7fb7dda72e00 1 --2- 172.21.15.96:0/1719061433 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb7bc036b30 0x7fb7bc039040 crc :-1 s=READY pgs=3834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.243 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.243+0000 7fb7dda72e00 1 -- 172.21.15.96:0/1719061433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589ad5f5df0 msgr2=0x5589ad5e1cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.243 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.243+0000 7fb7dda72e00 1 --2- 172.21.15.96:0/1719061433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589ad5f5df0 0x5589ad5e1cd0 crc :-1 s=READY pgs=12881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.243 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.243+0000 7fb7dda72e00 1 -- 172.21.15.96:0/1719061433 shutdown_connections 2024-03-21T06:36:11.243 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.243+0000 7fb7dda72e00 1 --2- 172.21.15.96:0/1719061433 >> v2:172.21.15.184:6811/276480837 conn(0x5589ad5f8d40 0x5589ad5e2720 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.243 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.243+0000 7fb7dda72e00 1 --2- 172.21.15.96:0/1719061433 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb7bc036b30 0x7fb7bc039040 unknown :-1 s=CLOSED pgs=3834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.243 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.243+0000 7fb7dda72e00 1 --2- 172.21.15.96:0/1719061433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589ad5f5df0 0x5589ad5e1cd0 unknown :-1 s=CLOSED pgs=12881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.243 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.243+0000 7fb7dda72e00 1 -- 172.21.15.96:0/1719061433 >> 172.21.15.96:0/1719061433 conn(0x5589ad470e00 msgr2=0x5589ad5de850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:11.243 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.243+0000 7fb7dda72e00 1 -- 172.21.15.96:0/1719061433 shutdown_connections 2024-03-21T06:36:11.243 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.243+0000 7fb7dda72e00 1 -- 172.21.15.96:0/1719061433 wait complete. 2024-03-21T06:36:11.244 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:36:11.244 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 13dbd6ab-5566-4934-957c-dcb5f3ce8145 put obj10 /etc/passwd 2024-03-21T06:36:11.274 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.274+0000 7fc4e7c85e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.277 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.278+0000 7fc4e7c85e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.278 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.279+0000 7fc4e7c85e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:11.278 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.279+0000 7fc4e7c85e00 1 Processor -- start 2024-03-21T06:36:11.278 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.279+0000 7fc4e7c85e00 1 -- start start 2024-03-21T06:36:11.279 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.280+0000 7fc4e7c85e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fac7dd8a0 0x563fac9000b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.279 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.280+0000 7fc4e7c85e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fac77eea0 con 0x563fac7dd8a0 2024-03-21T06:36:11.280 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.280+0000 7fc4dbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fac7dd8a0 0x563fac9000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.280 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.280+0000 7fc4dbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fac7dd8a0 0x563fac9000b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33386/0 (socket says 172.21.15.96:33386) 2024-03-21T06:36:11.280 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.280+0000 7fc4dbfff640 1 -- 172.21.15.96:0/902299626 learned_addr learned my addr 172.21.15.96:0/902299626 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:11.280 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.280+0000 7fc4dbfff640 1 -- 172.21.15.96:0/902299626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fac721a30 con 0x563fac7dd8a0 2024-03-21T06:36:11.280 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.280+0000 7fc4dbfff640 1 --2- 172.21.15.96:0/902299626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fac7dd8a0 0x563fac9000b0 crc :-1 s=READY pgs=12884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93bf3ccf91682593 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.280 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.280+0000 7fc4da7fc640 1 -- 172.21.15.96:0/902299626 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc4cc02ecb0 con 0x563fac7dd8a0 2024-03-21T06:36:11.280 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.280+0000 7fc4da7fc640 1 -- 172.21.15.96:0/902299626 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc4cc02ee10 con 0x563fac7dd8a0 2024-03-21T06:36:11.280 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.280+0000 7fc4da7fc640 1 -- 172.21.15.96:0/902299626 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc4cc00a170 con 0x563fac7dd8a0 2024-03-21T06:36:11.283 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.281+0000 7fc4e7c85e00 1 -- 172.21.15.96:0/902299626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fac7dd8a0 msgr2=0x563fac9000b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.283 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.281+0000 7fc4e7c85e00 1 --2- 172.21.15.96:0/902299626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fac7dd8a0 0x563fac9000b0 crc :-1 s=READY pgs=12884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.283 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.281+0000 7fc4e7c85e00 1 -- 172.21.15.96:0/902299626 shutdown_connections 2024-03-21T06:36:11.283 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.281+0000 7fc4e7c85e00 1 --2- 172.21.15.96:0/902299626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fac7dd8a0 0x563fac9000b0 unknown :-1 s=CLOSED pgs=12884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.283 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.281+0000 7fc4e7c85e00 1 -- 172.21.15.96:0/902299626 >> 172.21.15.96:0/902299626 conn(0x563fac6f3e00 msgr2=0x563fac874e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:11.283 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.281+0000 7fc4e7c85e00 1 -- 172.21.15.96:0/902299626 shutdown_connections 2024-03-21T06:36:11.283 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.281+0000 7fc4e7c85e00 1 -- 172.21.15.96:0/902299626 wait complete. 2024-03-21T06:36:11.283 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.281+0000 7fc4e7c85e00 1 Processor -- start 2024-03-21T06:36:11.283 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.281+0000 7fc4e7c85e00 1 -- start start 2024-03-21T06:36:11.284 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.282+0000 7fc4e7c85e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fac7dd8a0 0x563fac90bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.284 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.282+0000 7fc4e7c85e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fac71f550 con 0x563fac7dd8a0 2024-03-21T06:36:11.284 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.282+0000 7fc4dbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fac7dd8a0 0x563fac90bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.284 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.282+0000 7fc4dbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fac7dd8a0 0x563fac90bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33400/0 (socket says 172.21.15.96:33400) 2024-03-21T06:36:11.284 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.282+0000 7fc4dbfff640 1 -- 172.21.15.96:0/1712229081 learned_addr learned my addr 172.21.15.96:0/1712229081 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:11.284 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.282+0000 7fc4dbfff640 1 -- 172.21.15.96:0/1712229081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fac714680 con 0x563fac7dd8a0 2024-03-21T06:36:11.284 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.283+0000 7fc4dbfff640 1 --2- 172.21.15.96:0/1712229081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fac7dd8a0 0x563fac90bd40 crc :-1 s=READY pgs=12885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.284 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.283+0000 7fc4d97fa640 1 -- 172.21.15.96:0/1712229081 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc4cc03d040 con 0x563fac7dd8a0 2024-03-21T06:36:11.284 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.283+0000 7fc4d97fa640 1 -- 172.21.15.96:0/1712229081 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc4cc037430 con 0x563fac7dd8a0 2024-03-21T06:36:11.284 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.283+0000 7fc4e7c85e00 1 -- 172.21.15.96:0/1712229081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563fac7140e0 con 0x563fac7dd8a0 2024-03-21T06:36:11.284 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.283+0000 7fc4d97fa640 1 -- 172.21.15.96:0/1712229081 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc4cc0376d0 con 0x563fac7dd8a0 2024-03-21T06:36:11.284 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.283+0000 7fc4e7c85e00 1 -- 172.21.15.96:0/1712229081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fac862620 con 0x563fac7dd8a0 2024-03-21T06:36:11.284 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.283+0000 7fc4d97fa640 1 -- 172.21.15.96:0/1712229081 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fc4cc0378d0 con 0x563fac7dd8a0 2024-03-21T06:36:11.284 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.284+0000 7fc4d97fa640 1 --2- 172.21.15.96:0/1712229081 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc4c8036b30 0x7fc4c8039040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.285 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.284+0000 7fc4d97fa640 1 -- 172.21.15.96:0/1712229081 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(686..686 src has 402..686) v4 ==== 3791+0+0 (crc 0 0 0) 0x7fc4cc06d280 con 0x563fac7dd8a0 2024-03-21T06:36:11.285 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.284+0000 7fc4e7c85e00 1 --2- 172.21.15.96:0/1712229081 >> v2:172.21.15.96:6801/932229279 conn(0x563fac877860 0x563fac860fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:11.285 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.284+0000 7fc4e7c85e00 1 -- 172.21.15.96:0/1712229081 --> v2:172.21.15.96:6801/932229279 -- osd_op(unknown.0.0:1 200.b 200:d1337354:::obj10:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e686) v9 -- 0x563fac863070 con 0x563fac877860 2024-03-21T06:36:11.285 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.284+0000 7fc4db7fe640 1 --2- 172.21.15.96:0/1712229081 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc4c8036b30 0x7fc4c8039040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:11.285 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.284+0000 7fc4e099b640 1 --2- 172.21.15.96:0/1712229081 >> v2:172.21.15.96:6801/932229279 conn(0x563fac877860 0x563fac860fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:36:11.285 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.284+0000 7fc4db7fe640 1 --2- 172.21.15.96:0/1712229081 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc4c8036b30 0x7fc4c8039040 crc :-1 s=READY pgs=3835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.285 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.286+0000 7fc4e099b640 1 --2- 172.21.15.96:0/1712229081 >> v2:172.21.15.96:6801/932229279 conn(0x563fac877860 0x563fac860fd0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:11.293 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.293+0000 7fc4e099b640 1 -- 172.21.15.96:0/1712229081 <== osd.0 v2:172.21.15.96:6801/932229279 1 ==== osd_op_reply(1 obj10 [writefull 0~11812] v686'1 uv1 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fc4d400a430 con 0x563fac877860 2024-03-21T06:36:11.293 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.293+0000 7fc4e7c85e00 1 -- 172.21.15.96:0/1712229081 >> v2:172.21.15.96:6801/932229279 conn(0x563fac877860 msgr2=0x563fac860fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.293 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.293+0000 7fc4e7c85e00 1 --2- 172.21.15.96:0/1712229081 >> v2:172.21.15.96:6801/932229279 conn(0x563fac877860 0x563fac860fd0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.294 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.293+0000 7fc4e7c85e00 1 -- 172.21.15.96:0/1712229081 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc4c8036b30 msgr2=0x7fc4c8039040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.294 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.293+0000 7fc4e7c85e00 1 --2- 172.21.15.96:0/1712229081 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc4c8036b30 0x7fc4c8039040 crc :-1 s=READY pgs=3835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.294 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.293+0000 7fc4e7c85e00 1 -- 172.21.15.96:0/1712229081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fac7dd8a0 msgr2=0x563fac90bd40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:11.294 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.293+0000 7fc4e7c85e00 1 --2- 172.21.15.96:0/1712229081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fac7dd8a0 0x563fac90bd40 crc :-1 s=READY pgs=12885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.294 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.293+0000 7fc4e7c85e00 1 -- 172.21.15.96:0/1712229081 shutdown_connections 2024-03-21T06:36:11.294 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.293+0000 7fc4e7c85e00 1 --2- 172.21.15.96:0/1712229081 >> v2:172.21.15.96:6801/932229279 conn(0x563fac877860 0x563fac860fd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.294 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.293+0000 7fc4e7c85e00 1 --2- 172.21.15.96:0/1712229081 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc4c8036b30 0x7fc4c8039040 unknown :-1 s=CLOSED pgs=3835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.294 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.293+0000 7fc4e7c85e00 1 --2- 172.21.15.96:0/1712229081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fac7dd8a0 0x563fac90bd40 unknown :-1 s=CLOSED pgs=12885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:11.294 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.293+0000 7fc4e7c85e00 1 -- 172.21.15.96:0/1712229081 >> 172.21.15.96:0/1712229081 conn(0x563fac6f3e00 msgr2=0x563fac874e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:11.294 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.293+0000 7fc4e7c85e00 1 -- 172.21.15.96:0/1712229081 shutdown_connections 2024-03-21T06:36:11.294 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:11.293+0000 7fc4e7c85e00 1 -- 172.21.15.96:0/1712229081 wait complete. 2024-03-21T06:36:11.295 INFO:tasks.workunit.client.0.smithi096.stderr:+ sleep 30 2024-03-21T06:36:41.297 INFO:tasks.workunit.client.0.smithi096.stderr:+ pid=98816 2024-03-21T06:36:41.298 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph osd pool set-quota 13dbd6ab-5566-4934-957c-dcb5f3ce8145 max_objects 100 2024-03-21T06:36:41.298 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 13dbd6ab-5566-4934-957c-dcb5f3ce8145 put onemore /etc/passwd 2024-03-21T06:36:41.334 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.335+0000 7f03f782fe00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:41.339 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.340+0000 7f03f782fe00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:41.339 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.340+0000 7f03f782fe00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:41.340 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.341+0000 7f03f782fe00 1 Processor -- start 2024-03-21T06:36:41.340 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.341+0000 7f03f782fe00 1 -- start start 2024-03-21T06:36:41.342 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.342+0000 7f03f782fe00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be55cc05a0 0x55be55cd4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:41.342 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.342+0000 7f03f782fe00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be55c73ea0 con 0x55be55cc05a0 2024-03-21T06:36:41.342 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.342+0000 7f03f0849640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be55cc05a0 0x55be55cd4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:41.342 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.342+0000 7f03f0849640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be55cc05a0 0x55be55cd4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55618/0 (socket says 172.21.15.96:55618) 2024-03-21T06:36:41.342 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.342+0000 7f03f0849640 1 -- 172.21.15.96:0/1831945929 learned_addr learned my addr 172.21.15.96:0/1831945929 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:41.342 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.342+0000 7f03f0849640 1 -- 172.21.15.96:0/1831945929 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be55c16a30 con 0x55be55cc05a0 2024-03-21T06:36:41.342 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.343+0000 7f03f0849640 1 --2- 172.21.15.96:0/1831945929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be55cc05a0 0x55be55cd4b50 crc :-1 s=READY pgs=12888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0666a89e644475 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:41.343 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.343+0000 7f03eaffd640 1 -- 172.21.15.96:0/1831945929 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f03dc00d6b0 con 0x55be55cc05a0 2024-03-21T06:36:41.343 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.343+0000 7f03eaffd640 1 -- 172.21.15.96:0/1831945929 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f03dc00d810 con 0x55be55cc05a0 2024-03-21T06:36:41.343 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.343+0000 7f03eaffd640 1 -- 172.21.15.96:0/1831945929 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f03dc00dab0 con 0x55be55cc05a0 2024-03-21T06:36:41.344 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.344+0000 7f03f782fe00 1 -- 172.21.15.96:0/1831945929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be55cc05a0 msgr2=0x55be55cd4b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:41.344 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.344+0000 7f03f782fe00 1 --2- 172.21.15.96:0/1831945929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be55cc05a0 0x55be55cd4b50 crc :-1 s=READY pgs=12888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:41.344 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.344+0000 7f03f782fe00 1 -- 172.21.15.96:0/1831945929 shutdown_connections 2024-03-21T06:36:41.344 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.344+0000 7f03f782fe00 1 --2- 172.21.15.96:0/1831945929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be55cc05a0 0x55be55cd4b50 unknown :-1 s=CLOSED pgs=12888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:41.344 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.344+0000 7f03f782fe00 1 -- 172.21.15.96:0/1831945929 >> 172.21.15.96:0/1831945929 conn(0x55be55be8e00 msgr2=0x55be55d68ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:41.344 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.344+0000 7f03f782fe00 1 -- 172.21.15.96:0/1831945929 shutdown_connections 2024-03-21T06:36:41.344 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.344+0000 7f03f782fe00 1 -- 172.21.15.96:0/1831945929 wait complete. 2024-03-21T06:36:41.344 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.345+0000 7f03f782fe00 1 Processor -- start 2024-03-21T06:36:41.344 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.345+0000 7f03f782fe00 1 -- start start 2024-03-21T06:36:41.346 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.345+0000 7f03f782fe00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be55cc05a0 0x55be55e009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:41.346 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.345+0000 7f03f782fe00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be55c14550 con 0x55be55cc05a0 2024-03-21T06:36:41.346 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.345+0000 7f03f0849640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be55cc05a0 0x55be55e009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:41.346 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.345+0000 7f03f0849640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be55cc05a0 0x55be55e009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55620/0 (socket says 172.21.15.96:55620) 2024-03-21T06:36:41.346 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.345+0000 7f03f0849640 1 -- 172.21.15.96:0/842441806 learned_addr learned my addr 172.21.15.96:0/842441806 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:41.346 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.346+0000 7f03f0849640 1 -- 172.21.15.96:0/842441806 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be55c09680 con 0x55be55cc05a0 2024-03-21T06:36:41.346 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.346+0000 7f03f0849640 1 --2- 172.21.15.96:0/842441806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be55cc05a0 0x55be55e009b0 crc :-1 s=READY pgs=12889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:41.346 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.346+0000 7f03e9ffb640 1 -- 172.21.15.96:0/842441806 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f03dc012040 con 0x55be55cc05a0 2024-03-21T06:36:41.347 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.346+0000 7f03e9ffb640 1 -- 172.21.15.96:0/842441806 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f03dc013040 con 0x55be55cc05a0 2024-03-21T06:36:41.347 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.346+0000 7f03f782fe00 1 -- 172.21.15.96:0/842441806 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55be55c090e0 con 0x55be55cc05a0 2024-03-21T06:36:41.347 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.346+0000 7f03e9ffb640 1 -- 172.21.15.96:0/842441806 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f03dc00ac70 con 0x55be55cc05a0 2024-03-21T06:36:41.347 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.346+0000 7f03f782fe00 1 -- 172.21.15.96:0/842441806 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be55cd88c0 con 0x55be55cc05a0 2024-03-21T06:36:41.347 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.347+0000 7f03e9ffb640 1 -- 172.21.15.96:0/842441806 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f03dc0093a0 con 0x55be55cc05a0 2024-03-21T06:36:41.348 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.347+0000 7f03e9ffb640 1 --2- 172.21.15.96:0/842441806 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f03d8036b80 0x7f03d8039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:41.348 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.347+0000 7f03e9ffb640 1 -- 172.21.15.96:0/842441806 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(689..689 src has 402..689) v4 ==== 3791+0+0 (crc 0 0 0) 0x7f03dc049a10 con 0x55be55cc05a0 2024-03-21T06:36:41.348 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.347+0000 7f03e9ffb640 1 -- 172.21.15.96:0/842441806 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=690}) v3 -- 0x7f03d803cd90 con 0x55be55cc05a0 2024-03-21T06:36:41.348 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.348+0000 7f03f782fe00 1 --2- 172.21.15.96:0/842441806 >> v2:172.21.15.96:6801/932229279 conn(0x55be55d699c0 0x55be55d55f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:41.348 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.348+0000 7f03ebfff640 1 --2- 172.21.15.96:0/842441806 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f03d8036b80 0x7f03d8039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:41.348 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.348+0000 7f03f104a640 1 --2- 172.21.15.96:0/842441806 >> v2:172.21.15.96:6801/932229279 conn(0x55be55d699c0 0x55be55d55f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:36:41.348 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.348+0000 7f03ebfff640 1 --2- 172.21.15.96:0/842441806 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f03d8036b80 0x7f03d8039090 crc :-1 s=READY pgs=3836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:41.349 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.350+0000 7f03f104a640 1 --2- 172.21.15.96:0/842441806 >> v2:172.21.15.96:6801/932229279 conn(0x55be55d699c0 0x55be55d55f10 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:41.398 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.391+0000 7fb2ce59c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:41.398 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.399+0000 7fb2ce59c640 1 Processor -- start 2024-03-21T06:36:41.398 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.399+0000 7fb2ce59c640 1 -- start start 2024-03-21T06:36:41.400 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.400+0000 7fb2ce59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb2c8134280 0x7fb2c81366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:41.400 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.400+0000 7fb2ce59c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c8130f70 con 0x7fb2c81314b0 2024-03-21T06:36:41.400 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.400+0000 7fb2ce59c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c81311a0 con 0x7fb2c8134280 2024-03-21T06:36:41.400 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.400+0000 7fb2c7fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fb2c81314b0 0x7fb2c81318c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:49344/0 (socket says 172.21.15.96:49344) 2024-03-21T06:36:41.400 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.400+0000 7fb2c7fff640 1 -- 172.21.15.96:0/1336156494 learned_addr learned my addr 172.21.15.96:0/1336156494 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:36:41.400 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.400+0000 7fb2c77fe640 1 --2- 172.21.15.96:0/1336156494 >> v2:172.21.15.96:3300/0 conn(0x7fb2c8134280 0x7fb2c81366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:41.400 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.401+0000 7fb2c77fe640 1 -- 172.21.15.96:0/1336156494 >> v1:172.21.15.96:6789/0 conn(0x7fb2c81314b0 legacy=0x7fb2c81318c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:41.400 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.401+0000 7fb2c77fe640 1 -- 172.21.15.96:0/1336156494 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c806ee10 con 0x7fb2c8134280 2024-03-21T06:36:41.400 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.401+0000 7fb2c6ffd640 1 -- 172.21.15.96:0/1336156494 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1800406805 0 0) 0x7fb2c8130f70 con 0x7fb2c81314b0 2024-03-21T06:36:41.401 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.401+0000 7fb2c77fe640 1 --2- 172.21.15.96:0/1336156494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2c8134280 0x7fb2c81366a0 crc :-1 s=READY pgs=12891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9e650fc9822dac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:41.401 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.401+0000 7fb2c6ffd640 1 -- 172.21.15.96:0/1336156494 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb2bc009ce0 con 0x7fb2c8134280 2024-03-21T06:36:41.401 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.401+0000 7fb2c6ffd640 1 -- 172.21.15.96:0/1336156494 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb2bc009e40 con 0x7fb2c8134280 2024-03-21T06:36:41.401 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.401+0000 7fb2c6ffd640 1 -- 172.21.15.96:0/1336156494 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb2bc00e360 con 0x7fb2c8134280 2024-03-21T06:36:41.402 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.402+0000 7fb2ce59c640 1 -- 172.21.15.96:0/1336156494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2c8134280 msgr2=0x7fb2c81366a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:41.402 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.402+0000 7fb2ce59c640 1 --2- 172.21.15.96:0/1336156494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2c8134280 0x7fb2c81366a0 crc :-1 s=READY pgs=12891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:41.402 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.402+0000 7fb2ce59c640 1 -- 172.21.15.96:0/1336156494 shutdown_connections 2024-03-21T06:36:41.402 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.402+0000 7fb2ce59c640 1 --2- 172.21.15.96:0/1336156494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2c8134280 0x7fb2c81366a0 unknown :-1 s=CLOSED pgs=12891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:41.402 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.402+0000 7fb2ce59c640 1 -- 172.21.15.96:0/1336156494 >> 172.21.15.96:0/1336156494 conn(0x7fb2c80a05c0 msgr2=0x7fb2c80a09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:41.402 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.402+0000 7fb2ce59c640 1 -- 172.21.15.96:0/1336156494 shutdown_connections 2024-03-21T06:36:41.402 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.402+0000 7fb2ce59c640 1 -- 172.21.15.96:0/1336156494 wait complete. 2024-03-21T06:36:41.402 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.402+0000 7fb2ce59c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:41.402 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.403+0000 7fb2ce59c640 1 Processor -- start 2024-03-21T06:36:41.402 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.403+0000 7fb2ce59c640 1 -- start start 2024-03-21T06:36:41.405 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.404+0000 7fb2ce59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2c81314b0 0x7fb2c81c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:41.405 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.404+0000 7fb2ce59c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c8122750 con 0x7fb2c81314b0 2024-03-21T06:36:41.405 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.404+0000 7fb2c7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2c81314b0 0x7fb2c81c3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:41.405 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.404+0000 7fb2c7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2c81314b0 0x7fb2c81c3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55632/0 (socket says 172.21.15.96:55632) 2024-03-21T06:36:41.405 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.404+0000 7fb2c7fff640 1 -- 172.21.15.96:0/1804310266 learned_addr learned my addr 172.21.15.96:0/1804310266 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:41.405 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.404+0000 7fb2c7fff640 1 -- 172.21.15.96:0/1804310266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c8120340 con 0x7fb2c81314b0 2024-03-21T06:36:41.405 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.404+0000 7fb2c7fff640 1 --2- 172.21.15.96:0/1804310266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2c81314b0 0x7fb2c81c3af0 crc :-1 s=READY pgs=12892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:41.405 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.404+0000 7fb2c4ff9640 1 -- 172.21.15.96:0/1804310266 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb2b8004af0 con 0x7fb2c81314b0 2024-03-21T06:36:41.405 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.404+0000 7fb2c4ff9640 1 -- 172.21.15.96:0/1804310266 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb2b8004c50 con 0x7fb2c81314b0 2024-03-21T06:36:41.405 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.404+0000 7fb2c4ff9640 1 -- 172.21.15.96:0/1804310266 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb2b8007120 con 0x7fb2c81314b0 2024-03-21T06:36:41.405 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.404+0000 7fb2ce59c640 1 -- 172.21.15.96:0/1804310266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c811fb30 con 0x7fb2c81314b0 2024-03-21T06:36:41.405 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.404+0000 7fb2ce59c640 1 -- 172.21.15.96:0/1804310266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c8121900 con 0x7fb2c81314b0 2024-03-21T06:36:41.405 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.405+0000 7fb2c4ff9640 1 -- 172.21.15.96:0/1804310266 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fb2b80072d0 con 0x7fb2c81314b0 2024-03-21T06:36:41.406 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.405+0000 7fb2ce59c640 1 -- 172.21.15.96:0/1804310266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c8120cf0 con 0x7fb2c81314b0 2024-03-21T06:36:41.406 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.405+0000 7fb2c4ff9640 1 --2- 172.21.15.96:0/1804310266 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb2b4038260 0x7fb2b403a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:41.406 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.405+0000 7fb2c4ff9640 1 -- 172.21.15.96:0/1804310266 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(689..689 src has 402..689) v4 ==== 3791+0+0 (crc 0 0 0) 0x7fb2b8047e30 con 0x7fb2c81314b0 2024-03-21T06:36:41.406 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.406+0000 7fb2c4ff9640 1 -- 172.21.15.96:0/1804310266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=690}) v3 -- 0x7fb2b403e470 con 0x7fb2c81314b0 2024-03-21T06:36:41.406 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.406+0000 7fb2c77fe640 1 --2- 172.21.15.96:0/1804310266 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb2b4038260 0x7fb2b403a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:41.406 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.406+0000 7fb2c77fe640 1 --2- 172.21.15.96:0/1804310266 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb2b4038260 0x7fb2b403a770 crc :-1 s=READY pgs=3837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:41.409 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.410+0000 7fb2c4ff9640 1 -- 172.21.15.96:0/1804310266 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fb2c8120cf0 con 0x7fb2c81314b0 2024-03-21T06:36:41.585 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.586+0000 7fb2ce59c640 1 -- 172.21.15.96:0/1804310266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7fb2c8120cf0 con 0x7fb2c81314b0 2024-03-21T06:36:41.880 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.881+0000 7fb2c4ff9640 1 -- 172.21.15.96:0/1804310266 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 v690) v1 ==== 225+0+0 (crc 0 0 0) 0x7fb2c8120cf0 con 0x7fb2c81314b0 2024-03-21T06:36:41.881 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.881+0000 7fb2c4ff9640 1 -- 172.21.15.96:0/1804310266 <== mon.0 v2:172.21.15.96:3300/0 8 ==== osd_map(690..690 src has 402..690) v4 ==== 628+0+0 (crc 0 0 0) 0x7fb2b80060b0 con 0x7fb2c81314b0 2024-03-21T06:36:41.881 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.881+0000 7f03e9ffb640 1 -- 172.21.15.96:0/842441806 <== mon.0 v2:172.21.15.96:3300/0 6 ==== osd_map(690..690 src has 402..690) v4 ==== 628+0+0 (crc 0 0 0) 0x7f03dc013a40 con 0x55be55cc05a0 2024-03-21T06:36:41.881 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.882+0000 7fb2c4ff9640 1 -- 172.21.15.96:0/1804310266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=691}) v3 -- 0x7fb2b403f4d0 con 0x7fb2c81314b0 2024-03-21T06:36:41.882 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.882+0000 7f03e9ffb640 1 -- 172.21.15.96:0/842441806 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=691}) v3 -- 0x7f03d803d420 con 0x55be55cc05a0 2024-03-21T06:36:41.988 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:41.989+0000 7fb2ce59c640 1 -- 172.21.15.96:0/1804310266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7fb2c8120cf0 con 0x7fb2c81314b0 2024-03-21T06:36:42.443 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.444+0000 7fb2c4ff9640 1 -- 172.21.15.96:0/1804310266 <== mon.0 v2:172.21.15.96:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 v691) v1 ==== 225+0+0 (crc 0 0 0) 0x7fb2c8120cf0 con 0x7fb2c81314b0 2024-03-21T06:36:42.443 INFO:tasks.workunit.client.0.smithi096.stderr:set-quota max_objects = 100 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 2024-03-21T06:36:42.444 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.444+0000 7fb2c4ff9640 1 -- 172.21.15.96:0/1804310266 <== mon.0 v2:172.21.15.96:3300/0 10 ==== osd_map(691..691 src has 402..691) v4 ==== 628+0+0 (crc 0 0 0) 0x7fb2b800eb10 con 0x7fb2c81314b0 2024-03-21T06:36:42.444 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.444+0000 7fb2c4ff9640 1 -- 172.21.15.96:0/1804310266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=692}) v3 -- 0x7fb2b403f710 con 0x7fb2c81314b0 2024-03-21T06:36:42.444 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.444+0000 7f03e9ffb640 1 -- 172.21.15.96:0/842441806 <== mon.0 v2:172.21.15.96:3300/0 7 ==== osd_map(691..691 src has 402..691) v4 ==== 628+0+0 (crc 0 0 0) 0x7f03dc013c50 con 0x55be55cc05a0 2024-03-21T06:36:42.444 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.444+0000 7f03e9ffb640 1 -- 172.21.15.96:0/842441806 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=692}) v3 -- 0x7f03d803d640 con 0x55be55cc05a0 2024-03-21T06:36:42.444 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.444+0000 7f03e9ffb640 1 -- 172.21.15.96:0/842441806 --> v2:172.21.15.96:6801/932229279 -- osd_op(unknown.0.0:1 200.5 200:b9ff7440:::onemore:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e691) v9 -- 0x7f03d803d860 con 0x55be55d699c0 2024-03-21T06:36:42.446 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.446+0000 7fb2ce59c640 1 -- 172.21.15.96:0/1804310266 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb2b4038260 msgr2=0x7fb2b403a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:42.446 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.446+0000 7fb2ce59c640 1 --2- 172.21.15.96:0/1804310266 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb2b4038260 0x7fb2b403a770 crc :-1 s=READY pgs=3837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.446 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.446+0000 7fb2ce59c640 1 -- 172.21.15.96:0/1804310266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2c81314b0 msgr2=0x7fb2c81c3af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:42.446 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.446+0000 7fb2ce59c640 1 --2- 172.21.15.96:0/1804310266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2c81314b0 0x7fb2c81c3af0 crc :-1 s=READY pgs=12892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.446 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.446+0000 7fb2ce59c640 1 -- 172.21.15.96:0/1804310266 shutdown_connections 2024-03-21T06:36:42.446 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.446+0000 7fb2ce59c640 1 --2- 172.21.15.96:0/1804310266 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb2b4038260 0x7fb2b403a770 unknown :-1 s=CLOSED pgs=3837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.446 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.446+0000 7fb2ce59c640 1 --2- 172.21.15.96:0/1804310266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2c81314b0 0x7fb2c81c3af0 unknown :-1 s=CLOSED pgs=12892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.446 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.446+0000 7fb2ce59c640 1 -- 172.21.15.96:0/1804310266 >> 172.21.15.96:0/1804310266 conn(0x7fb2c80a05c0 msgr2=0x7fb2c8132770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:42.446 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.446+0000 7fb2ce59c640 1 -- 172.21.15.96:0/1804310266 shutdown_connections 2024-03-21T06:36:42.446 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.446+0000 7fb2ce59c640 1 -- 172.21.15.96:0/1804310266 wait complete. 2024-03-21T06:36:42.457 INFO:tasks.workunit.client.0.smithi096.stderr:+ wait 98816 2024-03-21T06:36:42.612 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.611+0000 7f03f104a640 1 -- 172.21.15.96:0/842441806 <== osd.0 v2:172.21.15.96:6801/932229279 1 ==== osd_op_reply(1 onemore [writefull 0~11812] v691'2 uv2 ondisk = 0) v8 ==== 151+0+0 (crc 0 0 0) 0x7f03e400a430 con 0x55be55d699c0 2024-03-21T06:36:42.612 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.611+0000 7f03f782fe00 1 -- 172.21.15.96:0/842441806 >> v2:172.21.15.96:6801/932229279 conn(0x55be55d699c0 msgr2=0x55be55d55f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:42.612 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.611+0000 7f03f782fe00 1 --2- 172.21.15.96:0/842441806 >> v2:172.21.15.96:6801/932229279 conn(0x55be55d699c0 0x55be55d55f10 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.612 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.612+0000 7f03f782fe00 1 -- 172.21.15.96:0/842441806 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f03d8036b80 msgr2=0x7f03d8039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:42.612 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.612+0000 7f03f782fe00 1 --2- 172.21.15.96:0/842441806 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f03d8036b80 0x7f03d8039090 crc :-1 s=READY pgs=3836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.612 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.612+0000 7f03f782fe00 1 -- 172.21.15.96:0/842441806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be55cc05a0 msgr2=0x55be55e009b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:42.612 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.612+0000 7f03f782fe00 1 --2- 172.21.15.96:0/842441806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be55cc05a0 0x55be55e009b0 crc :-1 s=READY pgs=12889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.612 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.612+0000 7f03f782fe00 1 -- 172.21.15.96:0/842441806 shutdown_connections 2024-03-21T06:36:42.612 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.612+0000 7f03f782fe00 1 --2- 172.21.15.96:0/842441806 >> v2:172.21.15.96:6801/932229279 conn(0x55be55d699c0 0x55be55d55f10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.612 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.612+0000 7f03f782fe00 1 --2- 172.21.15.96:0/842441806 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f03d8036b80 0x7f03d8039090 unknown :-1 s=CLOSED pgs=3836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.612 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.612+0000 7f03f782fe00 1 --2- 172.21.15.96:0/842441806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be55cc05a0 0x55be55e009b0 unknown :-1 s=CLOSED pgs=12889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.612 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.612+0000 7f03f782fe00 1 -- 172.21.15.96:0/842441806 >> 172.21.15.96:0/842441806 conn(0x55be55be8e00 msgr2=0x55be55d68ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:42.613 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.612+0000 7f03f782fe00 1 -- 172.21.15.96:0/842441806 shutdown_connections 2024-03-21T06:36:42.613 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.612+0000 7f03f782fe00 1 -- 172.21.15.96:0/842441806 wait complete. 2024-03-21T06:36:42.613 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -ne 0 ']' 2024-03-21T06:36:42.613 INFO:tasks.workunit.client.0.smithi096.stderr:+ true 2024-03-21T06:36:42.613 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 13dbd6ab-5566-4934-957c-dcb5f3ce8145 put twomore /etc/passwd 2024-03-21T06:36:42.642 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.643+0000 7fd184086e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:42.647 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.648+0000 7fd184086e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:42.647 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.648+0000 7fd184086e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:42.647 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.648+0000 7fd184086e00 1 Processor -- start 2024-03-21T06:36:42.647 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.648+0000 7fd184086e00 1 -- start start 2024-03-21T06:36:42.649 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.649+0000 7fd184086e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e490eda640 0x55e490ffbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:42.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.649+0000 7fd184086e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e490e74ea0 con 0x55e490eda640 2024-03-21T06:36:42.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.649+0000 7fd17d19a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e490eda640 0x55e490ffbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:42.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.649+0000 7fd17d19a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e490eda640 0x55e490ffbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55660/0 (socket says 172.21.15.96:55660) 2024-03-21T06:36:42.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.649+0000 7fd17d19a640 1 -- 172.21.15.96:0/423271448 learned_addr learned my addr 172.21.15.96:0/423271448 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:42.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.649+0000 7fd17d19a640 1 -- 172.21.15.96:0/423271448 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e490e17a30 con 0x55e490eda640 2024-03-21T06:36:42.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.649+0000 7fd17d19a640 1 --2- 172.21.15.96:0/423271448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e490eda640 0x55e490ffbab0 crc :-1 s=READY pgs=12895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96945c95f245eb5c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:42.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.649+0000 7fd16f7fe640 1 -- 172.21.15.96:0/423271448 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd174029d10 con 0x55e490eda640 2024-03-21T06:36:42.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.649+0000 7fd16f7fe640 1 -- 172.21.15.96:0/423271448 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd17400a040 con 0x55e490eda640 2024-03-21T06:36:42.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.649+0000 7fd16f7fe640 1 -- 172.21.15.96:0/423271448 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd17400a2e0 con 0x55e490eda640 2024-03-21T06:36:42.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.650+0000 7fd184086e00 1 -- 172.21.15.96:0/423271448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e490eda640 msgr2=0x55e490ffbab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:42.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.650+0000 7fd184086e00 1 --2- 172.21.15.96:0/423271448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e490eda640 0x55e490ffbab0 crc :-1 s=READY pgs=12895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.650+0000 7fd184086e00 1 -- 172.21.15.96:0/423271448 shutdown_connections 2024-03-21T06:36:42.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.650+0000 7fd184086e00 1 --2- 172.21.15.96:0/423271448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e490eda640 0x55e490ffbab0 unknown :-1 s=CLOSED pgs=12895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.650+0000 7fd184086e00 1 -- 172.21.15.96:0/423271448 >> 172.21.15.96:0/423271448 conn(0x55e490de9e00 msgr2=0x55e490ed8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:42.651 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.650+0000 7fd184086e00 1 -- 172.21.15.96:0/423271448 shutdown_connections 2024-03-21T06:36:42.651 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.650+0000 7fd184086e00 1 -- 172.21.15.96:0/423271448 wait complete. 2024-03-21T06:36:42.651 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.651+0000 7fd184086e00 1 Processor -- start 2024-03-21T06:36:42.651 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.651+0000 7fd184086e00 1 -- start start 2024-03-21T06:36:42.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.651+0000 7fd184086e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e490eda640 0x55e490ec2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:42.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.651+0000 7fd184086e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e490e15550 con 0x55e490eda640 2024-03-21T06:36:42.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.651+0000 7fd17d19a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e490eda640 0x55e490ec2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:42.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.651+0000 7fd17d19a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e490eda640 0x55e490ec2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55672/0 (socket says 172.21.15.96:55672) 2024-03-21T06:36:42.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.651+0000 7fd17d19a640 1 -- 172.21.15.96:0/137378056 learned_addr learned my addr 172.21.15.96:0/137378056 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:42.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.652+0000 7fd17d19a640 1 -- 172.21.15.96:0/137378056 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e490e0a680 con 0x55e490eda640 2024-03-21T06:36:42.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.652+0000 7fd17d19a640 1 --2- 172.21.15.96:0/137378056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e490eda640 0x55e490ec2f40 crc :-1 s=READY pgs=12896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:42.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.652+0000 7fd16e7fc640 1 -- 172.21.15.96:0/137378056 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd17400a2e0 con 0x55e490eda640 2024-03-21T06:36:42.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.652+0000 7fd16e7fc640 1 -- 172.21.15.96:0/137378056 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd174029b50 con 0x55e490eda640 2024-03-21T06:36:42.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.652+0000 7fd184086e00 1 -- 172.21.15.96:0/137378056 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e490e0a0e0 con 0x55e490eda640 2024-03-21T06:36:42.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.652+0000 7fd16e7fc640 1 -- 172.21.15.96:0/137378056 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd174035450 con 0x55e490eda640 2024-03-21T06:36:42.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.652+0000 7fd184086e00 1 -- 172.21.15.96:0/137378056 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e490f5faf0 con 0x55e490eda640 2024-03-21T06:36:42.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.653+0000 7fd16e7fc640 1 -- 172.21.15.96:0/137378056 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fd174035600 con 0x55e490eda640 2024-03-21T06:36:42.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.653+0000 7fd16e7fc640 1 --2- 172.21.15.96:0/137378056 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd15c036b80 0x7fd15c039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:42.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.653+0000 7fd16e7fc640 1 -- 172.21.15.96:0/137378056 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(691..691 src has 402..691) v4 ==== 3791+0+0 (crc 0 0 0) 0x7fd17402e040 con 0x55e490eda640 2024-03-21T06:36:42.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.653+0000 7fd184086e00 1 --2- 172.21.15.96:0/137378056 >> v2:172.21.15.96:6805/13988474 conn(0x55e490eddae0 0x55e4910085a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:42.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.653+0000 7fd184086e00 1 -- 172.21.15.96:0/137378056 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:1 200.6 200:7019d845:::twomore:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e691) v9 -- 0x55e490f5f680 con 0x55e490eddae0 2024-03-21T06:36:42.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.653+0000 7fd17c999640 1 --2- 172.21.15.96:0/137378056 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd15c036b80 0x7fd15c039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:42.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.654+0000 7fd17c999640 1 --2- 172.21.15.96:0/137378056 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd15c036b80 0x7fd15c039090 crc :-1 s=READY pgs=3838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:42.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.654+0000 7fd17d99b640 1 --2- 172.21.15.96:0/137378056 >> v2:172.21.15.96:6805/13988474 conn(0x55e490eddae0 0x55e4910085a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:36:42.655 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.656+0000 7fd17d99b640 1 --2- 172.21.15.96:0/137378056 >> v2:172.21.15.96:6805/13988474 conn(0x55e490eddae0 0x55e4910085a0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:42.778 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.777+0000 7fd17d99b640 1 -- 172.21.15.96:0/137378056 <== osd.1 v2:172.21.15.96:6805/13988474 1 ==== osd_op_reply(1 twomore [writefull 0~11812] v691'1 uv1 ondisk = 0) v8 ==== 151+0+0 (crc 0 0 0) 0x7fd17000a430 con 0x55e490eddae0 2024-03-21T06:36:42.778 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.777+0000 7fd184086e00 1 -- 172.21.15.96:0/137378056 >> v2:172.21.15.96:6805/13988474 conn(0x55e490eddae0 msgr2=0x55e4910085a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:42.778 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.777+0000 7fd184086e00 1 --2- 172.21.15.96:0/137378056 >> v2:172.21.15.96:6805/13988474 conn(0x55e490eddae0 0x55e4910085a0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.778 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.778+0000 7fd184086e00 1 -- 172.21.15.96:0/137378056 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd15c036b80 msgr2=0x7fd15c039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:42.778 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.778+0000 7fd184086e00 1 --2- 172.21.15.96:0/137378056 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd15c036b80 0x7fd15c039090 crc :-1 s=READY pgs=3838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.778 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.778+0000 7fd184086e00 1 -- 172.21.15.96:0/137378056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e490eda640 msgr2=0x55e490ec2f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:42.778 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.778+0000 7fd184086e00 1 --2- 172.21.15.96:0/137378056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e490eda640 0x55e490ec2f40 crc :-1 s=READY pgs=12896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.778 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.778+0000 7fd184086e00 1 -- 172.21.15.96:0/137378056 shutdown_connections 2024-03-21T06:36:42.778 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.778+0000 7fd184086e00 1 --2- 172.21.15.96:0/137378056 >> v2:172.21.15.96:6805/13988474 conn(0x55e490eddae0 0x55e4910085a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.779 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.778+0000 7fd184086e00 1 --2- 172.21.15.96:0/137378056 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd15c036b80 0x7fd15c039090 unknown :-1 s=CLOSED pgs=3838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.779 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.778+0000 7fd184086e00 1 --2- 172.21.15.96:0/137378056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e490eda640 0x55e490ec2f40 unknown :-1 s=CLOSED pgs=12896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.779 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.778+0000 7fd184086e00 1 -- 172.21.15.96:0/137378056 >> 172.21.15.96:0/137378056 conn(0x55e490de9e00 msgr2=0x55e490f73790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:42.779 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.778+0000 7fd184086e00 1 -- 172.21.15.96:0/137378056 shutdown_connections 2024-03-21T06:36:42.779 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.778+0000 7fd184086e00 1 -- 172.21.15.96:0/137378056 wait complete. 2024-03-21T06:36:42.779 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph osd pool set-quota 13dbd6ab-5566-4934-957c-dcb5f3ce8145 max_bytes 100 2024-03-21T06:36:42.866 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.860+0000 7f47b026d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:42.867 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.868+0000 7f47b026d640 1 Processor -- start 2024-03-21T06:36:42.867 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.868+0000 7f47b026d640 1 -- start start 2024-03-21T06:36:42.869 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.869+0000 7f47b026d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f47a81329f0 0x7f47a809d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:42.869 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.869+0000 7f47b026d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a8131f80 con 0x7f47a81325e0 2024-03-21T06:36:42.870 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.869+0000 7f47b026d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a8067f30 con 0x7f47a81329f0 2024-03-21T06:36:42.870 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.869+0000 7f47ad808640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f47a81329f0 0x7f47a809d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:42.870 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.869+0000 7f47ae009640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f47a81325e0 0x7f47a809ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:49370/0 (socket says 172.21.15.96:49370) 2024-03-21T06:36:42.870 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.869+0000 7f47ae009640 1 -- 172.21.15.96:0/1546933991 learned_addr learned my addr 172.21.15.96:0/1546933991 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:36:42.870 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.869+0000 7f47ad808640 1 -- 172.21.15.96:0/1546933991 >> v1:172.21.15.96:6789/0 conn(0x7f47a81325e0 legacy=0x7f47a809ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:42.870 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.869+0000 7f47ad808640 1 -- 172.21.15.96:0/1546933991 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a809e710 con 0x7f47a81329f0 2024-03-21T06:36:42.870 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.869+0000 7f47ad808640 1 --2- 172.21.15.96:0/1546933991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47a81329f0 0x7f47a809d140 crc :-1 s=READY pgs=12898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b31d9093e7ab1ec server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:42.870 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.869+0000 7f47ad007640 1 -- 172.21.15.96:0/1546933991 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f479c00e040 con 0x7f47a81329f0 2024-03-21T06:36:42.870 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.869+0000 7f47ad007640 1 -- 172.21.15.96:0/1546933991 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f479c007670 con 0x7f47a81329f0 2024-03-21T06:36:42.870 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.869+0000 7f47ad007640 1 -- 172.21.15.96:0/1546933991 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f479c007910 con 0x7f47a81329f0 2024-03-21T06:36:42.870 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.870+0000 7f47b026d640 1 -- 172.21.15.96:0/1546933991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47a81329f0 msgr2=0x7f47a809d140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:42.870 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.870+0000 7f47b026d640 1 --2- 172.21.15.96:0/1546933991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47a81329f0 0x7f47a809d140 crc :-1 s=READY pgs=12898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.870 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.870+0000 7f47b026d640 1 -- 172.21.15.96:0/1546933991 shutdown_connections 2024-03-21T06:36:42.871 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.870+0000 7f47b026d640 1 --2- 172.21.15.96:0/1546933991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47a81329f0 0x7f47a809d140 unknown :-1 s=CLOSED pgs=12898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:42.871 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.870+0000 7f47b026d640 1 -- 172.21.15.96:0/1546933991 >> 172.21.15.96:0/1546933991 conn(0x7f47a80a05c0 msgr2=0x7f47a80a09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:42.871 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.870+0000 7f47b026d640 1 -- 172.21.15.96:0/1546933991 shutdown_connections 2024-03-21T06:36:42.871 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.870+0000 7f47b026d640 1 -- 172.21.15.96:0/1546933991 wait complete. 2024-03-21T06:36:42.871 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.870+0000 7f47b026d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:36:42.873 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.871+0000 7f47b026d640 1 Processor -- start 2024-03-21T06:36:42.874 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.871+0000 7f47b026d640 1 -- start start 2024-03-21T06:36:42.874 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.872+0000 7f47b026d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47a81325e0 0x7f47a806dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:42.874 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.872+0000 7f47b026d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a806f650 con 0x7f47a81325e0 2024-03-21T06:36:42.874 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.872+0000 7f47ae009640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47a81325e0 0x7f47a806dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:42.874 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.872+0000 7f47ae009640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47a81325e0 0x7f47a806dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55684/0 (socket says 172.21.15.96:55684) 2024-03-21T06:36:42.874 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.872+0000 7f47ae009640 1 -- 172.21.15.96:0/2518553774 learned_addr learned my addr 172.21.15.96:0/2518553774 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:36:42.874 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.872+0000 7f47ae009640 1 -- 172.21.15.96:0/2518553774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a8123c20 con 0x7f47a81325e0 2024-03-21T06:36:42.874 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.872+0000 7f47ae009640 1 --2- 172.21.15.96:0/2518553774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47a81325e0 0x7f47a806dae0 crc :-1 s=READY pgs=12899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:42.874 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.873+0000 7f47a6ffd640 1 -- 172.21.15.96:0/2518553774 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f47a00059d0 con 0x7f47a81325e0 2024-03-21T06:36:42.874 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.873+0000 7f47a6ffd640 1 -- 172.21.15.96:0/2518553774 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f47a0005b30 con 0x7f47a81325e0 2024-03-21T06:36:42.874 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.873+0000 7f47a6ffd640 1 -- 172.21.15.96:0/2518553774 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f47a0005dd0 con 0x7f47a81325e0 2024-03-21T06:36:42.874 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.873+0000 7f47b026d640 1 -- 172.21.15.96:0/2518553774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a81226c0 con 0x7f47a81325e0 2024-03-21T06:36:42.874 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.873+0000 7f47b026d640 1 -- 172.21.15.96:0/2518553774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a8123940 con 0x7f47a81325e0 2024-03-21T06:36:42.875 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.874+0000 7f47a6ffd640 1 -- 172.21.15.96:0/2518553774 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f47a00070b0 con 0x7f47a81325e0 2024-03-21T06:36:42.875 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.874+0000 7f47b026d640 1 -- 172.21.15.96:0/2518553774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a8068e80 con 0x7f47a81325e0 2024-03-21T06:36:42.875 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.874+0000 7f47a6ffd640 1 --2- 172.21.15.96:0/2518553774 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4794038260 0x7f479403a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:36:42.875 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.874+0000 7f47a6ffd640 1 -- 172.21.15.96:0/2518553774 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(691..691 src has 402..691) v4 ==== 3791+0+0 (crc 0 0 0) 0x7f47a0047cb0 con 0x7f47a81325e0 2024-03-21T06:36:42.875 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.874+0000 7f47ad808640 1 --2- 172.21.15.96:0/2518553774 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4794038260 0x7f479403a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:36:42.875 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.875+0000 7f47ad808640 1 --2- 172.21.15.96:0/2518553774 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4794038260 0x7f479403a770 crc :-1 s=READY pgs=3839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:36:42.878 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:42.879+0000 7f47a6ffd640 1 -- 172.21.15.96:0/2518553774 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f47a000d330 con 0x7f47a81325e0 2024-03-21T06:36:43.054 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:43.055+0000 7f47b026d640 1 -- 172.21.15.96:0/2518553774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f47a806bd30 con 0x7f47a81325e0 2024-03-21T06:36:43.888 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:43.888+0000 7f47a6ffd640 1 -- 172.21.15.96:0/2518553774 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 v692) v1 ==== 221+0+0 (crc 0 0 0) 0x7f47a000d150 con 0x7f47a81325e0 2024-03-21T06:36:43.994 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:43.995+0000 7f47b026d640 1 -- 172.21.15.96:0/2518553774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f47a8131b10 con 0x7f47a81325e0 2024-03-21T06:36:44.890 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:44.891+0000 7f47a6ffd640 1 -- 172.21.15.96:0/2518553774 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 v693) v1 ==== 221+0+0 (crc 0 0 0) 0x7f47a000d150 con 0x7f47a81325e0 2024-03-21T06:36:44.890 INFO:tasks.workunit.client.0.smithi096.stderr:set-quota max_bytes = 100 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 2024-03-21T06:36:44.893 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:44.893+0000 7f47b026d640 1 -- 172.21.15.96:0/2518553774 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4794038260 msgr2=0x7f479403a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:44.893 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:44.893+0000 7f47b026d640 1 --2- 172.21.15.96:0/2518553774 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4794038260 0x7f479403a770 crc :-1 s=READY pgs=3839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:44.893 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:44.893+0000 7f47b026d640 1 -- 172.21.15.96:0/2518553774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47a81325e0 msgr2=0x7f47a806dae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:36:44.893 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:44.893+0000 7f47b026d640 1 --2- 172.21.15.96:0/2518553774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47a81325e0 0x7f47a806dae0 crc :-1 s=READY pgs=12899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:44.893 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:44.893+0000 7f47b026d640 1 -- 172.21.15.96:0/2518553774 shutdown_connections 2024-03-21T06:36:44.893 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:44.893+0000 7f47b026d640 1 --2- 172.21.15.96:0/2518553774 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4794038260 0x7f479403a770 unknown :-1 s=CLOSED pgs=3839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:44.893 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:44.893+0000 7f47b026d640 1 --2- 172.21.15.96:0/2518553774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47a81325e0 0x7f47a806dae0 unknown :-1 s=CLOSED pgs=12899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:36:44.893 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:44.893+0000 7f47b026d640 1 -- 172.21.15.96:0/2518553774 >> 172.21.15.96:0/2518553774 conn(0x7f47a80a05c0 msgr2=0x7f47a81b9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:36:44.893 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:44.893+0000 7f47b026d640 1 -- 172.21.15.96:0/2518553774 shutdown_connections 2024-03-21T06:36:44.893 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:36:44.893+0000 7f47b026d640 1 -- 172.21.15.96:0/2518553774 wait complete. 2024-03-21T06:36:44.903 INFO:tasks.workunit.client.0.smithi096.stderr:+ sleep 30 2024-03-21T06:37:14.905 INFO:tasks.workunit.client.0.smithi096.stderr:+ pid=98907 2024-03-21T06:37:14.905 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph osd pool set-quota 13dbd6ab-5566-4934-957c-dcb5f3ce8145 max_bytes 0 2024-03-21T06:37:14.905 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 13dbd6ab-5566-4934-957c-dcb5f3ce8145 put two /etc/passwd 2024-03-21T06:37:14.942 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.943+0000 7f433ba32e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:14.947 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.948+0000 7f433ba32e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:14.947 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.948+0000 7f433ba32e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:14.947 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.948+0000 7f433ba32e00 1 Processor -- start 2024-03-21T06:37:14.948 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.948+0000 7f433ba32e00 1 -- start start 2024-03-21T06:37:14.949 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.949+0000 7f433ba32e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56391ab4f7e0 0x56391ac6f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:14.949 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.949+0000 7f433ba32e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x56391aaeeea0 con 0x56391ab4f7e0 2024-03-21T06:37:14.949 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.949+0000 7f4334b9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56391ab4f7e0 0x56391ac6f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:14.949 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.949+0000 7f4334b9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56391ab4f7e0 0x56391ac6f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42418/0 (socket says 172.21.15.96:42418) 2024-03-21T06:37:14.949 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.949+0000 7f4334b9a640 1 -- 172.21.15.96:0/810176149 learned_addr learned my addr 172.21.15.96:0/810176149 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:14.949 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.950+0000 7f4334b9a640 1 -- 172.21.15.96:0/810176149 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56391aa91a30 con 0x56391ab4f7e0 2024-03-21T06:37:14.950 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.950+0000 7f4334b9a640 1 --2- 172.21.15.96:0/810176149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56391ab4f7e0 0x56391ac6f960 crc :-1 s=READY pgs=12902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1454c651421f39 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:14.950 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.950+0000 7f432effd640 1 -- 172.21.15.96:0/810176149 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4320033040 con 0x56391ab4f7e0 2024-03-21T06:37:14.950 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.950+0000 7f432effd640 1 -- 172.21.15.96:0/810176149 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4320034040 con 0x56391ab4f7e0 2024-03-21T06:37:14.950 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.950+0000 7f432effd640 1 -- 172.21.15.96:0/810176149 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f432000a2a0 con 0x56391ab4f7e0 2024-03-21T06:37:14.952 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.951+0000 7f433ba32e00 1 -- 172.21.15.96:0/810176149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56391ab4f7e0 msgr2=0x56391ac6f960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:14.952 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.951+0000 7f433ba32e00 1 --2- 172.21.15.96:0/810176149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56391ab4f7e0 0x56391ac6f960 crc :-1 s=READY pgs=12902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:14.952 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.951+0000 7f433ba32e00 1 -- 172.21.15.96:0/810176149 shutdown_connections 2024-03-21T06:37:14.952 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.951+0000 7f433ba32e00 1 --2- 172.21.15.96:0/810176149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56391ab4f7e0 0x56391ac6f960 unknown :-1 s=CLOSED pgs=12902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:14.952 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.951+0000 7f433ba32e00 1 -- 172.21.15.96:0/810176149 >> 172.21.15.96:0/810176149 conn(0x56391aa63e00 msgr2=0x56391abe3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:14.952 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.951+0000 7f433ba32e00 1 -- 172.21.15.96:0/810176149 shutdown_connections 2024-03-21T06:37:14.952 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.951+0000 7f433ba32e00 1 -- 172.21.15.96:0/810176149 wait complete. 2024-03-21T06:37:14.952 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.951+0000 7f433ba32e00 1 Processor -- start 2024-03-21T06:37:14.952 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.951+0000 7f433ba32e00 1 -- start start 2024-03-21T06:37:14.953 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.952+0000 7f433ba32e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56391ab4f7e0 0x56391ab52200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:14.953 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.952+0000 7f433ba32e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x56391aa8f550 con 0x56391ab4f7e0 2024-03-21T06:37:14.953 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.952+0000 7f4334b9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56391ab4f7e0 0x56391ab52200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:14.953 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.952+0000 7f4334b9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56391ab4f7e0 0x56391ab52200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42420/0 (socket says 172.21.15.96:42420) 2024-03-21T06:37:14.953 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.952+0000 7f4334b9a640 1 -- 172.21.15.96:0/2790501136 learned_addr learned my addr 172.21.15.96:0/2790501136 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:14.953 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.952+0000 7f4334b9a640 1 -- 172.21.15.96:0/2790501136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56391aa84680 con 0x56391ab4f7e0 2024-03-21T06:37:14.953 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.953+0000 7f4334b9a640 1 --2- 172.21.15.96:0/2790501136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56391ab4f7e0 0x56391ab52200 crc :-1 s=READY pgs=12903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:14.953 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.953+0000 7f432dffb640 1 -- 172.21.15.96:0/2790501136 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4320033040 con 0x56391ab4f7e0 2024-03-21T06:37:14.953 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.953+0000 7f432dffb640 1 -- 172.21.15.96:0/2790501136 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f432002da10 con 0x56391ab4f7e0 2024-03-21T06:37:14.953 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.953+0000 7f433ba32e00 1 -- 172.21.15.96:0/2790501136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56391aa840e0 con 0x56391ab4f7e0 2024-03-21T06:37:14.953 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.953+0000 7f432dffb640 1 -- 172.21.15.96:0/2790501136 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f432002dcb0 con 0x56391ab4f7e0 2024-03-21T06:37:14.953 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.953+0000 7f433ba32e00 1 -- 172.21.15.96:0/2790501136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56391ab54570 con 0x56391ab4f7e0 2024-03-21T06:37:14.954 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.954+0000 7f432dffb640 1 -- 172.21.15.96:0/2790501136 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f432003a3f0 con 0x56391ab4f7e0 2024-03-21T06:37:14.954 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.954+0000 7f432dffb640 1 --2- 172.21.15.96:0/2790501136 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f431c036b80 0x7f431c039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:14.955 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.954+0000 7f432dffb640 1 -- 172.21.15.96:0/2790501136 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(695..695 src has 402..695) v4 ==== 3791+0+0 (crc 0 0 0) 0x7f432006b180 con 0x56391ab4f7e0 2024-03-21T06:37:14.955 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.954+0000 7f432dffb640 1 -- 172.21.15.96:0/2790501136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=696}) v3 -- 0x7f431c03cd90 con 0x56391ab4f7e0 2024-03-21T06:37:14.955 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.954+0000 7f433ba32e00 1 --2- 172.21.15.96:0/2790501136 >> v2:172.21.15.96:6805/13988474 conn(0x56391ab4fcf0 0x56391ac7dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:14.955 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.954+0000 7f432ffff640 1 --2- 172.21.15.96:0/2790501136 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f431c036b80 0x7f431c039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:14.955 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.955+0000 7f432ffff640 1 --2- 172.21.15.96:0/2790501136 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f431c036b80 0x7f431c039090 crc :-1 s=READY pgs=3840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:14.955 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.955+0000 7f433539b640 1 --2- 172.21.15.96:0/2790501136 >> v2:172.21.15.96:6805/13988474 conn(0x56391ab4fcf0 0x56391ac7dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:37:14.956 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.957+0000 7f433539b640 1 --2- 172.21.15.96:0/2790501136 >> v2:172.21.15.96:6805/13988474 conn(0x56391ab4fcf0 0x56391ac7dcc0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:15.004 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:14.997+0000 7fa20c79c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:15.004 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.005+0000 7fa20c79c640 1 Processor -- start 2024-03-21T06:37:15.005 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.005+0000 7fa20c79c640 1 -- start start 2024-03-21T06:37:15.007 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.006+0000 7fa20c79c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa204134280 0x7fa2041366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:15.008 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.006+0000 7fa20c79c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa204130f70 con 0x7fa2041314b0 2024-03-21T06:37:15.008 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.006+0000 7fa20c79c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2041311a0 con 0x7fa204134280 2024-03-21T06:37:15.008 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.007+0000 7fa209d37640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa204134280 0x7fa2041366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:15.008 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.007+0000 7fa20a538640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fa2041314b0 0x7fa2041318c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33230/0 (socket says 172.21.15.96:33230) 2024-03-21T06:37:15.008 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.007+0000 7fa20a538640 1 -- 172.21.15.96:0/2944988929 learned_addr learned my addr 172.21.15.96:0/2944988929 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:37:15.008 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.007+0000 7fa209d37640 1 -- 172.21.15.96:0/2944988929 >> v1:172.21.15.96:6789/0 conn(0x7fa2041314b0 legacy=0x7fa2041318c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:15.008 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.007+0000 7fa209d37640 1 -- 172.21.15.96:0/2944988929 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20406ee10 con 0x7fa204134280 2024-03-21T06:37:15.008 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.007+0000 7fa209536640 1 -- 172.21.15.96:0/2944988929 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132463150 0 0) 0x7fa204130f70 con 0x7fa2041314b0 2024-03-21T06:37:15.008 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.007+0000 7fa209d37640 1 --2- 172.21.15.96:0/2944988929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa204134280 0x7fa2041366a0 crc :-1 s=READY pgs=12905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2040f4ba10029026 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:15.008 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.007+0000 7fa209536640 1 -- 172.21.15.96:0/2944988929 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa1f4009ce0 con 0x7fa204134280 2024-03-21T06:37:15.008 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.007+0000 7fa209536640 1 -- 172.21.15.96:0/2944988929 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa1f4009e40 con 0x7fa204134280 2024-03-21T06:37:15.008 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.007+0000 7fa209536640 1 -- 172.21.15.96:0/2944988929 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa1f400e360 con 0x7fa204134280 2024-03-21T06:37:15.008 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.008+0000 7fa20c79c640 1 -- 172.21.15.96:0/2944988929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa204134280 msgr2=0x7fa2041366a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:15.009 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.008+0000 7fa20c79c640 1 --2- 172.21.15.96:0/2944988929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa204134280 0x7fa2041366a0 crc :-1 s=READY pgs=12905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:15.009 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.008+0000 7fa20c79c640 1 -- 172.21.15.96:0/2944988929 shutdown_connections 2024-03-21T06:37:15.009 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.008+0000 7fa20c79c640 1 --2- 172.21.15.96:0/2944988929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa204134280 0x7fa2041366a0 unknown :-1 s=CLOSED pgs=12905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:15.009 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.008+0000 7fa20c79c640 1 -- 172.21.15.96:0/2944988929 >> 172.21.15.96:0/2944988929 conn(0x7fa2040a05c0 msgr2=0x7fa2040a09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:15.009 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.008+0000 7fa20c79c640 1 -- 172.21.15.96:0/2944988929 shutdown_connections 2024-03-21T06:37:15.009 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.008+0000 7fa20c79c640 1 -- 172.21.15.96:0/2944988929 wait complete. 2024-03-21T06:37:15.009 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.008+0000 7fa20c79c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:15.011 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.009+0000 7fa20c79c640 1 Processor -- start 2024-03-21T06:37:15.012 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.009+0000 7fa20c79c640 1 -- start start 2024-03-21T06:37:15.012 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.010+0000 7fa20c79c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2041314b0 0x7fa204120ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:15.012 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.010+0000 7fa20c79c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa204124c90 con 0x7fa2041314b0 2024-03-21T06:37:15.012 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.010+0000 7fa20a538640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2041314b0 0x7fa204120ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:15.012 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.010+0000 7fa20a538640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2041314b0 0x7fa204120ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42430/0 (socket says 172.21.15.96:42430) 2024-03-21T06:37:15.012 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.010+0000 7fa20a538640 1 -- 172.21.15.96:0/1978357300 learned_addr learned my addr 172.21.15.96:0/1978357300 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:15.012 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.010+0000 7fa20a538640 1 -- 172.21.15.96:0/1978357300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa204121b80 con 0x7fa2041314b0 2024-03-21T06:37:15.012 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.010+0000 7fa20a538640 1 --2- 172.21.15.96:0/1978357300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2041314b0 0x7fa204120ae0 crc :-1 s=READY pgs=12906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:15.012 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.010+0000 7fa1faffd640 1 -- 172.21.15.96:0/1978357300 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa200004af0 con 0x7fa2041314b0 2024-03-21T06:37:15.012 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.010+0000 7fa1faffd640 1 -- 172.21.15.96:0/1978357300 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fa200004c50 con 0x7fa2041314b0 2024-03-21T06:37:15.012 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.010+0000 7fa20c79c640 1 -- 172.21.15.96:0/1978357300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa204130be0 con 0x7fa2041314b0 2024-03-21T06:37:15.012 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.010+0000 7fa1faffd640 1 -- 172.21.15.96:0/1978357300 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fa200007120 con 0x7fa2041314b0 2024-03-21T06:37:15.012 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.010+0000 7fa20c79c640 1 -- 172.21.15.96:0/1978357300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa204123f60 con 0x7fa2041314b0 2024-03-21T06:37:15.012 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.011+0000 7fa1faffd640 1 -- 172.21.15.96:0/1978357300 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fa2000072d0 con 0x7fa2041314b0 2024-03-21T06:37:15.013 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.011+0000 7fa20c79c640 1 -- 172.21.15.96:0/1978357300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa204124410 con 0x7fa2041314b0 2024-03-21T06:37:15.013 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.012+0000 7fa1faffd640 1 --2- 172.21.15.96:0/1978357300 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa1e8038260 0x7fa1e803a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:15.013 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.012+0000 7fa1faffd640 1 -- 172.21.15.96:0/1978357300 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(695..695 src has 402..695) v4 ==== 3791+0+0 (crc 0 0 0) 0x7fa200047d70 con 0x7fa2041314b0 2024-03-21T06:37:15.013 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.012+0000 7fa1faffd640 1 -- 172.21.15.96:0/1978357300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=696}) v3 -- 0x7fa1e803e470 con 0x7fa2041314b0 2024-03-21T06:37:15.013 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.012+0000 7fa209d37640 1 --2- 172.21.15.96:0/1978357300 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa1e8038260 0x7fa1e803a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:15.013 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.012+0000 7fa209d37640 1 --2- 172.21.15.96:0/1978357300 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa1e8038260 0x7fa1e803a770 crc :-1 s=READY pgs=3841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:15.015 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.017+0000 7fa1faffd640 1 -- 172.21.15.96:0/1978357300 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fa20000f230 con 0x7fa2041314b0 2024-03-21T06:37:15.188 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.189+0000 7fa20c79c640 1 -- 172.21.15.96:0/1978357300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fa204123340 con 0x7fa2041314b0 2024-03-21T06:37:15.890 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.891+0000 7fa1faffd640 1 -- 172.21.15.96:0/1978357300 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 v696) v1 ==== 217+0+0 (crc 0 0 0) 0x7fa20000a5d0 con 0x7fa2041314b0 2024-03-21T06:37:15.892 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.893+0000 7fa1faffd640 1 -- 172.21.15.96:0/1978357300 <== mon.0 v2:172.21.15.96:3300/0 8 ==== osd_map(696..696 src has 402..696) v4 ==== 628+0+0 (crc 0 0 0) 0x7fa20000eb10 con 0x7fa2041314b0 2024-03-21T06:37:15.892 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.893+0000 7f432dffb640 1 -- 172.21.15.96:0/2790501136 <== mon.0 v2:172.21.15.96:3300/0 6 ==== osd_map(696..696 src has 402..696) v4 ==== 628+0+0 (crc 0 0 0) 0x7f4320038df0 con 0x56391ab4f7e0 2024-03-21T06:37:15.893 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.893+0000 7fa1faffd640 1 -- 172.21.15.96:0/1978357300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=697}) v3 -- 0x7fa1e803f460 con 0x7fa2041314b0 2024-03-21T06:37:15.893 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.893+0000 7f432dffb640 1 -- 172.21.15.96:0/2790501136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=697}) v3 -- 0x7f431c03d420 con 0x56391ab4f7e0 2024-03-21T06:37:15.994 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:15.995+0000 7fa20c79c640 1 -- 172.21.15.96:0/1978357300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fa204068e80 con 0x7fa2041314b0 2024-03-21T06:37:16.896 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:16.897+0000 7fa1faffd640 1 -- 172.21.15.96:0/1978357300 <== mon.0 v2:172.21.15.96:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 v697) v1 ==== 217+0+0 (crc 0 0 0) 0x7fa20000a7b0 con 0x7fa2041314b0 2024-03-21T06:37:16.896 INFO:tasks.workunit.client.0.smithi096.stderr:set-quota max_bytes = 0 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 2024-03-21T06:37:16.899 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:16.899+0000 7fa20c79c640 1 -- 172.21.15.96:0/1978357300 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa1e8038260 msgr2=0x7fa1e803a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:16.899 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:16.899+0000 7fa20c79c640 1 --2- 172.21.15.96:0/1978357300 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa1e8038260 0x7fa1e803a770 crc :-1 s=READY pgs=3841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:16.899 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:16.899+0000 7fa20c79c640 1 -- 172.21.15.96:0/1978357300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2041314b0 msgr2=0x7fa204120ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:16.899 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:16.899+0000 7fa20c79c640 1 --2- 172.21.15.96:0/1978357300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2041314b0 0x7fa204120ae0 crc :-1 s=READY pgs=12906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:16.899 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:16.899+0000 7fa1faffd640 1 -- 172.21.15.96:0/1978357300 <== mon.0 v2:172.21.15.96:3300/0 10 ==== osd_map(697..697 src has 402..697) v4 ==== 628+0+0 (crc 0 0 0) 0x7fa20000a3a0 con 0x7fa2041314b0 2024-03-21T06:37:16.899 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:16.899+0000 7f432dffb640 1 -- 172.21.15.96:0/2790501136 <== mon.0 v2:172.21.15.96:3300/0 7 ==== osd_map(697..697 src has 402..697) v4 ==== 628+0+0 (crc 0 0 0) 0x7f432003a6f0 con 0x56391ab4f7e0 2024-03-21T06:37:16.899 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:16.899+0000 7fa20c79c640 1 -- 172.21.15.96:0/1978357300 shutdown_connections 2024-03-21T06:37:16.899 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:16.899+0000 7fa20c79c640 1 --2- 172.21.15.96:0/1978357300 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fa1e8038260 0x7fa1e803a770 unknown :-1 s=CLOSED pgs=3841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:16.899 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:16.899+0000 7fa20c79c640 1 --2- 172.21.15.96:0/1978357300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2041314b0 0x7fa204120ae0 unknown :-1 s=CLOSED pgs=12906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:16.900 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:16.899+0000 7fa20c79c640 1 -- 172.21.15.96:0/1978357300 >> 172.21.15.96:0/1978357300 conn(0x7fa2040a05c0 msgr2=0x7fa20406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:16.900 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:16.899+0000 7f432dffb640 1 -- 172.21.15.96:0/2790501136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=698}) v3 -- 0x7f431c03d700 con 0x56391ab4f7e0 2024-03-21T06:37:16.900 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:16.899+0000 7fa20c79c640 1 -- 172.21.15.96:0/1978357300 shutdown_connections 2024-03-21T06:37:16.900 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:16.900+0000 7fa20c79c640 1 -- 172.21.15.96:0/1978357300 wait complete. 2024-03-21T06:37:16.910 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph osd pool set-quota 13dbd6ab-5566-4934-957c-dcb5f3ce8145 max_objects 0 2024-03-21T06:37:17.002 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:16.995+0000 7fcfb26ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:17.002 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.003+0000 7fcfb26ad640 1 Processor -- start 2024-03-21T06:37:17.003 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.003+0000 7fcfb26ad640 1 -- start start 2024-03-21T06:37:17.004 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.004+0000 7fcfb26ad640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcfac1329f0 0x7fcfac09d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:17.004 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.004+0000 7fcfb26ad640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac131f80 con 0x7fcfac1325e0 2024-03-21T06:37:17.004 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.004+0000 7fcfb26ad640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfac067f30 con 0x7fcfac1329f0 2024-03-21T06:37:17.004 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.004+0000 7fcfab7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcfac1329f0 0x7fcfac09d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:17.004 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.004+0000 7fcfabfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fcfac1325e0 0x7fcfac09ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:48000/0 (socket says 172.21.15.96:48000) 2024-03-21T06:37:17.004 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.004+0000 7fcfabfff640 1 -- 172.21.15.96:0/2837131772 learned_addr learned my addr 172.21.15.96:0/2837131772 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:37:17.004 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.004+0000 7fcfab7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcfac1329f0 0x7fcfac09d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41604/0 (socket says 172.21.15.96:41604) 2024-03-21T06:37:17.004 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.005+0000 7fcfab7fe640 1 -- 172.21.15.96:0/2837131772 >> v1:172.21.15.96:6789/0 conn(0x7fcfac1325e0 legacy=0x7fcfac09ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:17.004 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.005+0000 7fcfab7fe640 1 -- 172.21.15.96:0/2837131772 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac09e710 con 0x7fcfac1329f0 2024-03-21T06:37:17.005 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.005+0000 7fcfaaffd640 1 -- 172.21.15.96:0/2837131772 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988575138 0 0) 0x7fcfac131f80 con 0x7fcfac1325e0 2024-03-21T06:37:17.005 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.005+0000 7fcfab7fe640 1 --2- 172.21.15.96:0/2837131772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfac1329f0 0x7fcfac09d140 crc :-1 s=READY pgs=12908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea33d55239286212 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:17.005 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.005+0000 7fcfaaffd640 1 -- 172.21.15.96:0/2837131772 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcfa0009ce0 con 0x7fcfac1329f0 2024-03-21T06:37:17.005 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.005+0000 7fcfaaffd640 1 -- 172.21.15.96:0/2837131772 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fcfa0009e40 con 0x7fcfac1329f0 2024-03-21T06:37:17.005 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.005+0000 7fcfaaffd640 1 -- 172.21.15.96:0/2837131772 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcfa000e360 con 0x7fcfac1329f0 2024-03-21T06:37:17.006 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.005+0000 7fcfb26ad640 1 -- 172.21.15.96:0/2837131772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfac1329f0 msgr2=0x7fcfac09d140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:17.006 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.005+0000 7fcfb26ad640 1 --2- 172.21.15.96:0/2837131772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfac1329f0 0x7fcfac09d140 crc :-1 s=READY pgs=12908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:17.006 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.006+0000 7fcfb26ad640 1 -- 172.21.15.96:0/2837131772 shutdown_connections 2024-03-21T06:37:17.006 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.006+0000 7fcfb26ad640 1 --2- 172.21.15.96:0/2837131772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfac1329f0 0x7fcfac09d140 unknown :-1 s=CLOSED pgs=12908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:17.006 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.006+0000 7fcfb26ad640 1 -- 172.21.15.96:0/2837131772 >> 172.21.15.96:0/2837131772 conn(0x7fcfac0a05c0 msgr2=0x7fcfac0a09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:17.006 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.006+0000 7fcfb26ad640 1 -- 172.21.15.96:0/2837131772 shutdown_connections 2024-03-21T06:37:17.007 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.006+0000 7fcfb26ad640 1 -- 172.21.15.96:0/2837131772 wait complete. 2024-03-21T06:37:17.007 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.006+0000 7fcfb26ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:17.007 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.007+0000 7fcfb26ad640 1 Processor -- start 2024-03-21T06:37:17.007 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.007+0000 7fcfb26ad640 1 -- start start 2024-03-21T06:37:17.009 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.007+0000 7fcfb26ad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfac1325e0 0x7fcfac1c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:17.009 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.008+0000 7fcfb26ad640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac11ff90 con 0x7fcfac1325e0 2024-03-21T06:37:17.009 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.008+0000 7fcfabfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfac1325e0 0x7fcfac1c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:17.009 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.008+0000 7fcfabfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfac1325e0 0x7fcfac1c37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41618/0 (socket says 172.21.15.96:41618) 2024-03-21T06:37:17.009 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.008+0000 7fcfabfff640 1 -- 172.21.15.96:0/2915736730 learned_addr learned my addr 172.21.15.96:0/2915736730 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:17.009 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.008+0000 7fcfabfff640 1 -- 172.21.15.96:0/2915736730 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac121880 con 0x7fcfac1325e0 2024-03-21T06:37:17.009 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.008+0000 7fcfabfff640 1 --2- 172.21.15.96:0/2915736730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfac1325e0 0x7fcfac1c37e0 crc :-1 s=READY pgs=12909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:17.009 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.008+0000 7fcfa8ff9640 1 -- 172.21.15.96:0/2915736730 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcf9c004af0 con 0x7fcfac1325e0 2024-03-21T06:37:17.010 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.008+0000 7fcfa8ff9640 1 -- 172.21.15.96:0/2915736730 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fcf9c004c50 con 0x7fcfac1325e0 2024-03-21T06:37:17.010 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.008+0000 7fcfb26ad640 1 -- 172.21.15.96:0/2915736730 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfac1c3440 con 0x7fcfac1325e0 2024-03-21T06:37:17.010 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.008+0000 7fcfa8ff9640 1 -- 172.21.15.96:0/2915736730 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcf9c007120 con 0x7fcfac1325e0 2024-03-21T06:37:17.010 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.008+0000 7fcfb26ad640 1 -- 172.21.15.96:0/2915736730 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfac1203c0 con 0x7fcfac1325e0 2024-03-21T06:37:17.010 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.009+0000 7fcfa8ff9640 1 -- 172.21.15.96:0/2915736730 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fcf9c0072d0 con 0x7fcfac1325e0 2024-03-21T06:37:17.010 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.009+0000 7fcfa8ff9640 1 --2- 172.21.15.96:0/2915736730 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcf98038260 0x7fcf9803a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:17.010 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.009+0000 7fcfa8ff9640 1 -- 172.21.15.96:0/2915736730 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(697..697 src has 402..697) v4 ==== 3791+0+0 (crc 0 0 0) 0x7fcf9c047f00 con 0x7fcfac1325e0 2024-03-21T06:37:17.010 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.009+0000 7fcfa8ff9640 1 -- 172.21.15.96:0/2915736730 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=698}) v3 -- 0x7fcf9803e470 con 0x7fcfac1325e0 2024-03-21T06:37:17.010 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.009+0000 7fcfb26ad640 1 -- 172.21.15.96:0/2915736730 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfac11fdb0 con 0x7fcfac1325e0 2024-03-21T06:37:17.010 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.009+0000 7fcfab7fe640 1 --2- 172.21.15.96:0/2915736730 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcf98038260 0x7fcf9803a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:17.010 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.010+0000 7fcfab7fe640 1 --2- 172.21.15.96:0/2915736730 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcf98038260 0x7fcf9803a770 crc :-1 s=READY pgs=3842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:17.013 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.014+0000 7fcfa8ff9640 1 -- 172.21.15.96:0/2915736730 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fcfac11fdb0 con 0x7fcfac1325e0 2024-03-21T06:37:17.189 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.190+0000 7fcfb26ad640 1 -- 172.21.15.96:0/2915736730 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fcfac11fdb0 con 0x7fcfac1325e0 2024-03-21T06:37:17.449 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.449+0000 7fcfa8ff9640 1 -- 172.21.15.96:0/2915736730 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 v698) v1 ==== 221+0+0 (crc 0 0 0) 0x7fcfac11fdb0 con 0x7fcfac1325e0 2024-03-21T06:37:17.451 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.452+0000 7f432dffb640 1 -- 172.21.15.96:0/2790501136 <== mon.0 v2:172.21.15.96:3300/0 8 ==== osd_map(698..698 src has 402..698) v4 ==== 628+0+0 (crc 0 0 0) 0x7f4320063900 con 0x56391ab4f7e0 2024-03-21T06:37:17.451 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.452+0000 7fcfa8ff9640 1 -- 172.21.15.96:0/2915736730 <== mon.0 v2:172.21.15.96:3300/0 8 ==== osd_map(698..698 src has 402..698) v4 ==== 628+0+0 (crc 0 0 0) 0x7fcf9c0060b0 con 0x7fcfac1325e0 2024-03-21T06:37:17.451 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.452+0000 7f432dffb640 1 -- 172.21.15.96:0/2790501136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=699}) v3 -- 0x7f431c03d700 con 0x56391ab4f7e0 2024-03-21T06:37:17.451 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.452+0000 7fcfa8ff9640 1 -- 172.21.15.96:0/2915736730 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=699}) v3 -- 0x7fcf9803f3b0 con 0x7fcfac1325e0 2024-03-21T06:37:17.451 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.452+0000 7f432dffb640 1 -- 172.21.15.96:0/2790501136 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:1 200.6 200:6b5d76a5:::two:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e698) v9 -- 0x7f431c03d950 con 0x56391ab4fcf0 2024-03-21T06:37:17.579 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.580+0000 7fcfb26ad640 1 -- 172.21.15.96:0/2915736730 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fcfac11fdb0 con 0x7fcfac1325e0 2024-03-21T06:37:17.630 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.630+0000 7f433539b640 1 -- 172.21.15.96:0/2790501136 <== osd.1 v2:172.21.15.96:6805/13988474 1 ==== osd_op_reply(1 two [writefull 0~11812] v698'2 uv2 ondisk = 0) v8 ==== 147+0+0 (crc 0 0 0) 0x7f432800a430 con 0x56391ab4fcf0 2024-03-21T06:37:17.630 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.630+0000 7f433ba32e00 1 -- 172.21.15.96:0/2790501136 >> v2:172.21.15.96:6805/13988474 conn(0x56391ab4fcf0 msgr2=0x56391ac7dcc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:17.630 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.630+0000 7f433ba32e00 1 --2- 172.21.15.96:0/2790501136 >> v2:172.21.15.96:6805/13988474 conn(0x56391ab4fcf0 0x56391ac7dcc0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:17.630 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.630+0000 7f433ba32e00 1 -- 172.21.15.96:0/2790501136 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f431c036b80 msgr2=0x7f431c039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:17.630 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.630+0000 7f433ba32e00 1 --2- 172.21.15.96:0/2790501136 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f431c036b80 0x7f431c039090 crc :-1 s=READY pgs=3840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:17.630 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.630+0000 7f433ba32e00 1 -- 172.21.15.96:0/2790501136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56391ab4f7e0 msgr2=0x56391ab52200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:17.630 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.630+0000 7f433ba32e00 1 --2- 172.21.15.96:0/2790501136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56391ab4f7e0 0x56391ab52200 crc :-1 s=READY pgs=12903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:17.630 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.630+0000 7f433ba32e00 1 -- 172.21.15.96:0/2790501136 shutdown_connections 2024-03-21T06:37:17.630 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.630+0000 7f433ba32e00 1 --2- 172.21.15.96:0/2790501136 >> v2:172.21.15.96:6805/13988474 conn(0x56391ab4fcf0 0x56391ac7dcc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:17.631 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.630+0000 7f433ba32e00 1 --2- 172.21.15.96:0/2790501136 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f431c036b80 0x7f431c039090 unknown :-1 s=CLOSED pgs=3840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:17.631 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.630+0000 7f433ba32e00 1 --2- 172.21.15.96:0/2790501136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56391ab4f7e0 0x56391ab52200 unknown :-1 s=CLOSED pgs=12903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:17.631 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.630+0000 7f433ba32e00 1 -- 172.21.15.96:0/2790501136 >> 172.21.15.96:0/2790501136 conn(0x56391aa63e00 msgr2=0x56391ab591f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:17.631 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.630+0000 7f433ba32e00 1 -- 172.21.15.96:0/2790501136 shutdown_connections 2024-03-21T06:37:17.631 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:17.630+0000 7f433ba32e00 1 -- 172.21.15.96:0/2790501136 wait complete. 2024-03-21T06:37:18.458 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.458+0000 7fcfa8ff9640 1 -- 172.21.15.96:0/2915736730 <== mon.0 v2:172.21.15.96:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 v699) v1 ==== 221+0+0 (crc 0 0 0) 0x7fcfac11fdb0 con 0x7fcfac1325e0 2024-03-21T06:37:18.458 INFO:tasks.workunit.client.0.smithi096.stderr:set-quota max_objects = 0 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 2024-03-21T06:37:18.460 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.460+0000 7fcfb26ad640 1 -- 172.21.15.96:0/2915736730 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcf98038260 msgr2=0x7fcf9803a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:18.460 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.460+0000 7fcfb26ad640 1 --2- 172.21.15.96:0/2915736730 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcf98038260 0x7fcf9803a770 crc :-1 s=READY pgs=3842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:18.460 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.460+0000 7fcfa8ff9640 1 -- 172.21.15.96:0/2915736730 <== mon.0 v2:172.21.15.96:3300/0 10 ==== osd_map(699..699 src has 402..699) v4 ==== 628+0+0 (crc 0 0 0) 0x7fcf9c00a3a0 con 0x7fcfac1325e0 2024-03-21T06:37:18.460 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.460+0000 7fcfb26ad640 1 -- 172.21.15.96:0/2915736730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfac1325e0 msgr2=0x7fcfac1c37e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:18.460 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.460+0000 7fcfb26ad640 1 --2- 172.21.15.96:0/2915736730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfac1325e0 0x7fcfac1c37e0 crc :-1 s=READY pgs=12909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:18.460 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.461+0000 7fcfb26ad640 1 -- 172.21.15.96:0/2915736730 shutdown_connections 2024-03-21T06:37:18.460 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.461+0000 7fcfb26ad640 1 --2- 172.21.15.96:0/2915736730 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcf98038260 0x7fcf9803a770 unknown :-1 s=CLOSED pgs=3842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:18.461 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.461+0000 7fcfb26ad640 1 --2- 172.21.15.96:0/2915736730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfac1325e0 0x7fcfac1c37e0 unknown :-1 s=CLOSED pgs=12909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:18.461 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.461+0000 7fcfb26ad640 1 -- 172.21.15.96:0/2915736730 >> 172.21.15.96:0/2915736730 conn(0x7fcfac0a05c0 msgr2=0x7fcfac1b9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:18.461 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.461+0000 7fcfb26ad640 1 -- 172.21.15.96:0/2915736730 shutdown_connections 2024-03-21T06:37:18.461 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.461+0000 7fcfb26ad640 1 -- 172.21.15.96:0/2915736730 wait complete. 2024-03-21T06:37:18.470 INFO:tasks.workunit.client.0.smithi096.stderr:+ wait 98907 2024-03-21T06:37:18.470 INFO:tasks.workunit.client.0.smithi096.stderr:+ '[' 0 -ne 0 ']' 2024-03-21T06:37:18.470 INFO:tasks.workunit.client.0.smithi096.stderr:+ true 2024-03-21T06:37:18.470 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 13dbd6ab-5566-4934-957c-dcb5f3ce8145 put three /etc/passwd 2024-03-21T06:37:18.502 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.502+0000 7fc3fe6dee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:18.506 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.507+0000 7fc3fe6dee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:18.506 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.507+0000 7fc3fe6dee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:18.506 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.507+0000 7fc3fe6dee00 1 Processor -- start 2024-03-21T06:37:18.506 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.507+0000 7fc3fe6dee00 1 -- start start 2024-03-21T06:37:18.509 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.508+0000 7fc3fe6dee00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560186abce30 0x560186b54860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:18.509 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.508+0000 7fc3fe6dee00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x560186a6fea0 con 0x560186abce30 2024-03-21T06:37:18.509 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.508+0000 7fc3f779a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560186abce30 0x560186b54860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:18.509 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.508+0000 7fc3f779a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560186abce30 0x560186b54860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41634/0 (socket says 172.21.15.96:41634) 2024-03-21T06:37:18.509 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.508+0000 7fc3f779a640 1 -- 172.21.15.96:0/1459603513 learned_addr learned my addr 172.21.15.96:0/1459603513 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:18.509 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.508+0000 7fc3f779a640 1 -- 172.21.15.96:0/1459603513 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560186a12a30 con 0x560186abce30 2024-03-21T06:37:18.509 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.508+0000 7fc3f779a640 1 --2- 172.21.15.96:0/1459603513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560186abce30 0x560186b54860 crc :-1 s=READY pgs=12912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a7b017af95ec5a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:18.509 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.508+0000 7fc3f5f97640 1 -- 172.21.15.96:0/1459603513 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc3ec033040 con 0x560186abce30 2024-03-21T06:37:18.509 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.508+0000 7fc3f5f97640 1 -- 172.21.15.96:0/1459603513 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc3ec034040 con 0x560186abce30 2024-03-21T06:37:18.509 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.508+0000 7fc3f5f97640 1 -- 172.21.15.96:0/1459603513 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc3ec00a170 con 0x560186abce30 2024-03-21T06:37:18.509 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.509+0000 7fc3fe6dee00 1 -- 172.21.15.96:0/1459603513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560186abce30 msgr2=0x560186b54860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:18.509 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.509+0000 7fc3fe6dee00 1 --2- 172.21.15.96:0/1459603513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560186abce30 0x560186b54860 crc :-1 s=READY pgs=12912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:18.509 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.509+0000 7fc3fe6dee00 1 -- 172.21.15.96:0/1459603513 shutdown_connections 2024-03-21T06:37:18.510 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.509+0000 7fc3fe6dee00 1 --2- 172.21.15.96:0/1459603513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560186abce30 0x560186b54860 unknown :-1 s=CLOSED pgs=12912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:18.510 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.509+0000 7fc3fe6dee00 1 -- 172.21.15.96:0/1459603513 >> 172.21.15.96:0/1459603513 conn(0x5601869e4e00 msgr2=0x560186aced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:18.510 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.509+0000 7fc3fe6dee00 1 -- 172.21.15.96:0/1459603513 shutdown_connections 2024-03-21T06:37:18.510 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.509+0000 7fc3fe6dee00 1 -- 172.21.15.96:0/1459603513 wait complete. 2024-03-21T06:37:18.510 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.510+0000 7fc3fe6dee00 1 Processor -- start 2024-03-21T06:37:18.510 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.510+0000 7fc3fe6dee00 1 -- start start 2024-03-21T06:37:18.512 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.510+0000 7fc3fe6dee00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560186abce30 0x560186b57d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:18.513 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.510+0000 7fc3fe6dee00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x560186a10550 con 0x560186abce30 2024-03-21T06:37:18.513 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.511+0000 7fc3f779a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560186abce30 0x560186b57d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:18.513 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.511+0000 7fc3f779a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560186abce30 0x560186b57d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41650/0 (socket says 172.21.15.96:41650) 2024-03-21T06:37:18.513 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.511+0000 7fc3f779a640 1 -- 172.21.15.96:0/2590201854 learned_addr learned my addr 172.21.15.96:0/2590201854 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:18.513 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.511+0000 7fc3f779a640 1 -- 172.21.15.96:0/2590201854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560186a05680 con 0x560186abce30 2024-03-21T06:37:18.513 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.511+0000 7fc3f779a640 1 --2- 172.21.15.96:0/2590201854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560186abce30 0x560186b57d60 crc :-1 s=READY pgs=12913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:18.513 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.511+0000 7fc3f4f95640 1 -- 172.21.15.96:0/2590201854 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc3ec02ed40 con 0x560186abce30 2024-03-21T06:37:18.513 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.511+0000 7fc3f4f95640 1 -- 172.21.15.96:0/2590201854 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc3ec0300b0 con 0x560186abce30 2024-03-21T06:37:18.513 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.511+0000 7fc3fe6dee00 1 -- 172.21.15.96:0/2590201854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560186a050e0 con 0x560186abce30 2024-03-21T06:37:18.513 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.511+0000 7fc3f4f95640 1 -- 172.21.15.96:0/2590201854 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc3ec02eea0 con 0x560186abce30 2024-03-21T06:37:18.513 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.511+0000 7fc3fe6dee00 1 -- 172.21.15.96:0/2590201854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560186adb5e0 con 0x560186abce30 2024-03-21T06:37:18.513 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.512+0000 7fc3f4f95640 1 -- 172.21.15.96:0/2590201854 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fc3ec0303a0 con 0x560186abce30 2024-03-21T06:37:18.513 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.512+0000 7fc3f4f95640 1 --2- 172.21.15.96:0/2590201854 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc3d8036b80 0x7fc3d8039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:18.513 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.512+0000 7fc3f4f95640 1 -- 172.21.15.96:0/2590201854 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(699..699 src has 402..699) v4 ==== 3791+0+0 (crc 0 0 0) 0x7fc3ec06aee0 con 0x560186abce30 2024-03-21T06:37:18.514 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.512+0000 7fc3fe6dee00 1 --2- 172.21.15.96:0/2590201854 >> v2:172.21.15.184:6811/276480837 conn(0x560186b69df0 0x560187002b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:18.514 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.512+0000 7fc3fe6dee00 1 -- 172.21.15.96:0/2590201854 --> v2:172.21.15.184:6811/276480837 -- osd_op(unknown.0.0:1 200.3 200:c57798d7:::three:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e699) v9 -- 0x560186b5b860 con 0x560186b69df0 2024-03-21T06:37:18.514 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.512+0000 7fc3f6f99640 1 --2- 172.21.15.96:0/2590201854 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc3d8036b80 0x7fc3d8039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:18.514 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.513+0000 7fc3f6f99640 1 --2- 172.21.15.96:0/2590201854 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc3d8036b80 0x7fc3d8039090 crc :-1 s=READY pgs=3843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:18.514 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.513+0000 7fc3f7f9b640 1 --2- 172.21.15.96:0/2590201854 >> v2:172.21.15.184:6811/276480837 conn(0x560186b69df0 0x560187002b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:37:18.514 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.515+0000 7fc3f7f9b640 1 --2- 172.21.15.96:0/2590201854 >> v2:172.21.15.184:6811/276480837 conn(0x560186b69df0 0x560187002b60 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:18.742 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.741+0000 7fc3f7f9b640 1 -- 172.21.15.96:0/2590201854 <== osd.3 v2:172.21.15.184:6811/276480837 1 ==== osd_op_reply(1 three [writefull 0~11812] v699'2 uv2 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fc3e800a430 con 0x560186b69df0 2024-03-21T06:37:18.743 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.741+0000 7fc3fe6dee00 1 -- 172.21.15.96:0/2590201854 >> v2:172.21.15.184:6811/276480837 conn(0x560186b69df0 msgr2=0x560187002b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:18.743 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.741+0000 7fc3fe6dee00 1 --2- 172.21.15.96:0/2590201854 >> v2:172.21.15.184:6811/276480837 conn(0x560186b69df0 0x560187002b60 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:18.743 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.741+0000 7fc3fe6dee00 1 -- 172.21.15.96:0/2590201854 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc3d8036b80 msgr2=0x7fc3d8039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:18.743 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.741+0000 7fc3fe6dee00 1 --2- 172.21.15.96:0/2590201854 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc3d8036b80 0x7fc3d8039090 crc :-1 s=READY pgs=3843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:18.743 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.741+0000 7fc3fe6dee00 1 -- 172.21.15.96:0/2590201854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560186abce30 msgr2=0x560186b57d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:18.743 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.741+0000 7fc3fe6dee00 1 --2- 172.21.15.96:0/2590201854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560186abce30 0x560186b57d60 crc :-1 s=READY pgs=12913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:18.743 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.741+0000 7fc3fe6dee00 1 -- 172.21.15.96:0/2590201854 shutdown_connections 2024-03-21T06:37:18.743 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.741+0000 7fc3fe6dee00 1 --2- 172.21.15.96:0/2590201854 >> v2:172.21.15.184:6811/276480837 conn(0x560186b69df0 0x560187002b60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:18.743 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.741+0000 7fc3fe6dee00 1 --2- 172.21.15.96:0/2590201854 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc3d8036b80 0x7fc3d8039090 unknown :-1 s=CLOSED pgs=3843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:18.743 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.741+0000 7fc3fe6dee00 1 --2- 172.21.15.96:0/2590201854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560186abce30 0x560186b57d60 unknown :-1 s=CLOSED pgs=12913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:18.744 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.741+0000 7fc3fe6dee00 1 -- 172.21.15.96:0/2590201854 >> 172.21.15.96:0/2590201854 conn(0x5601869e4e00 msgr2=0x560186aced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:18.744 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.742+0000 7fc3fe6dee00 1 -- 172.21.15.96:0/2590201854 shutdown_connections 2024-03-21T06:37:18.744 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.742+0000 7fc3fe6dee00 1 -- 172.21.15.96:0/2590201854 wait complete. 2024-03-21T06:37:18.744 INFO:tasks.workunit.client.0.smithi096.stderr:++ uuidgen 2024-03-21T06:37:18.744 INFO:tasks.workunit.client.0.smithi096.stderr:+ pp=416ac5b8-f35e-473b-b584-9498e7bbba57 2024-03-21T06:37:18.745 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph osd pool create 416ac5b8-f35e-473b-b584-9498e7bbba57 12 2024-03-21T06:37:18.834 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.828+0000 7f4c0679c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:18.835 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.836+0000 7f4c0679c640 1 Processor -- start 2024-03-21T06:37:18.835 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.836+0000 7f4c0679c640 1 -- start start 2024-03-21T06:37:18.837 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.837+0000 7f4c0679c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4c0009c360 0x7f4c0012d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:18.837 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.837+0000 7f4c0679c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0009c8e0 con 0x7f4c0012dc70 2024-03-21T06:37:18.837 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.837+0000 7f4c0679c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c0009cc30 con 0x7f4c0009c360 2024-03-21T06:37:18.838 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.837+0000 7f4bfffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4c0009c360 0x7f4c0012d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:18.838 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.837+0000 7f4bff7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f4c0012dc70 0x7f4c001306d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:48010/0 (socket says 172.21.15.96:48010) 2024-03-21T06:37:18.838 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.837+0000 7f4bff7fe640 1 -- 172.21.15.96:0/3935139913 learned_addr learned my addr 172.21.15.96:0/3935139913 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:37:18.838 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.837+0000 7f4bfffff640 1 -- 172.21.15.96:0/3935139913 >> v1:172.21.15.96:6789/0 conn(0x7f4c0012dc70 legacy=0x7f4c001306d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:18.838 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.837+0000 7f4bfffff640 1 -- 172.21.15.96:0/3935139913 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0012d2e0 con 0x7f4c0009c360 2024-03-21T06:37:18.838 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.837+0000 7f4bfeffd640 1 -- 172.21.15.96:0/3935139913 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920878782 0 0) 0x7f4c0009c8e0 con 0x7f4c0012dc70 2024-03-21T06:37:18.838 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.837+0000 7f4bfffff640 1 --2- 172.21.15.96:0/3935139913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c0009c360 0x7f4c0012d730 crc :-1 s=READY pgs=12915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5a3af8c843edbe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:18.838 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.837+0000 7f4bfeffd640 1 -- 172.21.15.96:0/3935139913 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4bf0009ce0 con 0x7f4c0009c360 2024-03-21T06:37:18.838 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.837+0000 7f4bfeffd640 1 -- 172.21.15.96:0/3935139913 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4bf0009e40 con 0x7f4c0009c360 2024-03-21T06:37:18.838 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.837+0000 7f4bfeffd640 1 -- 172.21.15.96:0/3935139913 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4bf000e360 con 0x7f4c0009c360 2024-03-21T06:37:18.838 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.838+0000 7f4c0679c640 1 -- 172.21.15.96:0/3935139913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c0009c360 msgr2=0x7f4c0012d730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:18.838 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.838+0000 7f4c0679c640 1 --2- 172.21.15.96:0/3935139913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c0009c360 0x7f4c0012d730 crc :-1 s=READY pgs=12915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:18.839 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.838+0000 7f4c0679c640 1 -- 172.21.15.96:0/3935139913 shutdown_connections 2024-03-21T06:37:18.839 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.838+0000 7f4c0679c640 1 --2- 172.21.15.96:0/3935139913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c0009c360 0x7f4c0012d730 unknown :-1 s=CLOSED pgs=12915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:18.839 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.838+0000 7f4c0679c640 1 -- 172.21.15.96:0/3935139913 >> 172.21.15.96:0/3935139913 conn(0x7f4c0006c890 msgr2=0x7f4c0006cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:18.839 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.838+0000 7f4c0679c640 1 -- 172.21.15.96:0/3935139913 shutdown_connections 2024-03-21T06:37:18.839 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.838+0000 7f4c0679c640 1 -- 172.21.15.96:0/3935139913 wait complete. 2024-03-21T06:37:18.839 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.838+0000 7f4c0679c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:18.841 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.839+0000 7f4c0679c640 1 Processor -- start 2024-03-21T06:37:18.841 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.839+0000 7f4c0679c640 1 -- start start 2024-03-21T06:37:18.841 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.840+0000 7f4c0679c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c0009c360 0x7f4c0011a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:18.841 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.840+0000 7f4c0679c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0011d2e0 con 0x7f4c0009c360 2024-03-21T06:37:18.841 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.840+0000 7f4bfffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c0009c360 0x7f4c0011a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:18.841 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.840+0000 7f4bfffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c0009c360 0x7f4c0011a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41672/0 (socket says 172.21.15.96:41672) 2024-03-21T06:37:18.841 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.840+0000 7f4bfffff640 1 -- 172.21.15.96:0/1669960939 learned_addr learned my addr 172.21.15.96:0/1669960939 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:18.841 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.840+0000 7f4bfffff640 1 -- 172.21.15.96:0/1669960939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0011bcd0 con 0x7f4c0009c360 2024-03-21T06:37:18.841 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.840+0000 7f4bfffff640 1 --2- 172.21.15.96:0/1669960939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c0009c360 0x7f4c0011a0b0 crc :-1 s=READY pgs=12916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:18.841 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.841+0000 7f4bfcff9640 1 -- 172.21.15.96:0/1669960939 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4bf0014540 con 0x7f4c0009c360 2024-03-21T06:37:18.842 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.841+0000 7f4bfcff9640 1 -- 172.21.15.96:0/1669960939 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4bf00146a0 con 0x7f4c0009c360 2024-03-21T06:37:18.842 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.841+0000 7f4c0679c640 1 -- 172.21.15.96:0/1669960939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0006a4c0 con 0x7f4c0009c360 2024-03-21T06:37:18.842 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.841+0000 7f4bfcff9640 1 -- 172.21.15.96:0/1669960939 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4bf0014940 con 0x7f4c0009c360 2024-03-21T06:37:18.842 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.841+0000 7f4c0679c640 1 -- 172.21.15.96:0/1669960939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0011c5b0 con 0x7f4c0009c360 2024-03-21T06:37:18.842 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.842+0000 7f4bfcff9640 1 -- 172.21.15.96:0/1669960939 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f4bf0014af0 con 0x7f4c0009c360 2024-03-21T06:37:18.842 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.842+0000 7f4c0679c640 1 -- 172.21.15.96:0/1669960939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c00068e80 con 0x7f4c0009c360 2024-03-21T06:37:18.842 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.842+0000 7f4bfcff9640 1 --2- 172.21.15.96:0/1669960939 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4bec038260 0x7f4bec03a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:18.842 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.842+0000 7f4bfcff9640 1 -- 172.21.15.96:0/1669960939 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(699..699 src has 402..699) v4 ==== 3791+0+0 (crc 0 0 0) 0x7f4bf0019040 con 0x7f4c0009c360 2024-03-21T06:37:18.842 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.842+0000 7f4bff7fe640 1 --2- 172.21.15.96:0/1669960939 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4bec038260 0x7f4bec03a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:18.842 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.842+0000 7f4bff7fe640 1 --2- 172.21.15.96:0/1669960939 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4bec038260 0x7f4bec03a770 crc :-1 s=READY pgs=3844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:18.846 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:18.847+0000 7f4bfcff9640 1 -- 172.21.15.96:0/1669960939 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f4bf0018190 con 0x7f4c0009c360 2024-03-21T06:37:19.018 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.019+0000 7f4c0679c640 1 -- 172.21.15.96:0/1669960939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "416ac5b8-f35e-473b-b584-9498e7bbba57", "pg_num": 12} v 0) v1 -- 0x7f4c0006bd30 con 0x7f4c0009c360 2024-03-21T06:37:19.461 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.462+0000 7f4bfcff9640 1 -- 172.21.15.96:0/1669960939 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "416ac5b8-f35e-473b-b584-9498e7bbba57", "pg_num": 12}]=0 pool '416ac5b8-f35e-473b-b584-9498e7bbba57' created v700) v1 ==== 176+0+0 (crc 0 0 0) 0x7f4bf000fb30 con 0x7f4c0009c360 2024-03-21T06:37:19.644 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.644+0000 7f4c0679c640 1 -- 172.21.15.96:0/1669960939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "416ac5b8-f35e-473b-b584-9498e7bbba57", "pg_num": 12} v 0) v1 -- 0x7f4c0011e0b0 con 0x7f4c0009c360 2024-03-21T06:37:19.644 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.645+0000 7f4bfcff9640 1 -- 172.21.15.96:0/1669960939 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "416ac5b8-f35e-473b-b584-9498e7bbba57", "pg_num": 12}]=0 pool '416ac5b8-f35e-473b-b584-9498e7bbba57' already exists v700) v1 ==== 183+0+0 (crc 0 0 0) 0x7f4bf000fd10 con 0x7f4c0009c360 2024-03-21T06:37:19.644 INFO:tasks.workunit.client.0.smithi096.stderr:pool '416ac5b8-f35e-473b-b584-9498e7bbba57' already exists 2024-03-21T06:37:19.646 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.647+0000 7f4c0679c640 1 -- 172.21.15.96:0/1669960939 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4bec038260 msgr2=0x7f4bec03a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:19.646 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.647+0000 7f4c0679c640 1 --2- 172.21.15.96:0/1669960939 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4bec038260 0x7f4bec03a770 crc :-1 s=READY pgs=3844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:19.646 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.647+0000 7f4c0679c640 1 -- 172.21.15.96:0/1669960939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c0009c360 msgr2=0x7f4c0011a0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:19.646 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.647+0000 7f4c0679c640 1 --2- 172.21.15.96:0/1669960939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c0009c360 0x7f4c0011a0b0 crc :-1 s=READY pgs=12916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:19.646 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.647+0000 7f4c0679c640 1 -- 172.21.15.96:0/1669960939 shutdown_connections 2024-03-21T06:37:19.647 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.647+0000 7f4c0679c640 1 --2- 172.21.15.96:0/1669960939 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4bec038260 0x7f4bec03a770 unknown :-1 s=CLOSED pgs=3844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:19.647 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.647+0000 7f4c0679c640 1 --2- 172.21.15.96:0/1669960939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c0009c360 0x7f4c0011a0b0 unknown :-1 s=CLOSED pgs=12916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:19.647 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.647+0000 7f4c0679c640 1 -- 172.21.15.96:0/1669960939 >> 172.21.15.96:0/1669960939 conn(0x7f4c0006c890 msgr2=0x7f4c0012c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:19.647 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.647+0000 7f4c0679c640 1 -- 172.21.15.96:0/1669960939 shutdown_connections 2024-03-21T06:37:19.647 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.647+0000 7f4c0679c640 1 -- 172.21.15.96:0/1669960939 wait complete. 2024-03-21T06:37:19.655 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph osd pool application enable 416ac5b8-f35e-473b-b584-9498e7bbba57 rados 2024-03-21T06:37:19.742 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.735+0000 7f7e7751c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:19.743 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.744+0000 7f7e7751c640 1 Processor -- start 2024-03-21T06:37:19.743 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.744+0000 7f7e7751c640 1 -- start start 2024-03-21T06:37:19.745 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.745+0000 7f7e7751c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7e701338e0 0x7f7e70135d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:19.745 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.745+0000 7f7e7751c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7009ed40 con 0x7f7e70130b10 2024-03-21T06:37:19.745 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.745+0000 7f7e7751c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e70067f10 con 0x7f7e701338e0 2024-03-21T06:37:19.745 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.745+0000 7f7e74ab7640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7e701338e0 0x7f7e70135d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:19.746 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.745+0000 7f7e752b8640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f7e70130b10 0x7f7e70130f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:48014/0 (socket says 172.21.15.96:48014) 2024-03-21T06:37:19.746 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.745+0000 7f7e752b8640 1 -- 172.21.15.96:0/1571623070 learned_addr learned my addr 172.21.15.96:0/1571623070 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:37:19.746 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.745+0000 7f7e74ab7640 1 -- 172.21.15.96:0/1571623070 >> v1:172.21.15.96:6789/0 conn(0x7f7e70130b10 legacy=0x7f7e70130f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:19.746 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.745+0000 7f7e74ab7640 1 -- 172.21.15.96:0/1571623070 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e701306e0 con 0x7f7e701338e0 2024-03-21T06:37:19.746 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.745+0000 7f7e74ab7640 1 --2- 172.21.15.96:0/1571623070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e701338e0 0x7f7e70135d00 crc :-1 s=READY pgs=12918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68633767f209ef62 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:19.746 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.745+0000 7f7e67fff640 1 -- 172.21.15.96:0/1571623070 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7e6000ac70 con 0x7f7e701338e0 2024-03-21T06:37:19.746 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.745+0000 7f7e67fff640 1 -- 172.21.15.96:0/1571623070 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7e6000add0 con 0x7f7e701338e0 2024-03-21T06:37:19.746 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.745+0000 7f7e67fff640 1 -- 172.21.15.96:0/1571623070 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7e6000f470 con 0x7f7e701338e0 2024-03-21T06:37:19.746 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.746+0000 7f7e7751c640 1 -- 172.21.15.96:0/1571623070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e701338e0 msgr2=0x7f7e70135d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:19.746 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.746+0000 7f7e7751c640 1 --2- 172.21.15.96:0/1571623070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e701338e0 0x7f7e70135d00 crc :-1 s=READY pgs=12918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:19.746 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.746+0000 7f7e7751c640 1 -- 172.21.15.96:0/1571623070 shutdown_connections 2024-03-21T06:37:19.746 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.746+0000 7f7e7751c640 1 --2- 172.21.15.96:0/1571623070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e701338e0 0x7f7e70135d00 unknown :-1 s=CLOSED pgs=12918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:19.746 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.746+0000 7f7e7751c640 1 -- 172.21.15.96:0/1571623070 >> 172.21.15.96:0/1571623070 conn(0x7f7e700a05c0 msgr2=0x7f7e700a09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:19.747 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.746+0000 7f7e7751c640 1 -- 172.21.15.96:0/1571623070 shutdown_connections 2024-03-21T06:37:19.747 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.746+0000 7f7e7751c640 1 -- 172.21.15.96:0/1571623070 wait complete. 2024-03-21T06:37:19.747 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.746+0000 7f7e7751c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:19.749 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.747+0000 7f7e7751c640 1 Processor -- start 2024-03-21T06:37:19.749 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.747+0000 7f7e7751c640 1 -- start start 2024-03-21T06:37:19.749 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.748+0000 7f7e7751c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e70130b10 0x7f7e701c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:19.749 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.748+0000 7f7e7751c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7011aa30 con 0x7f7e70130b10 2024-03-21T06:37:19.749 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.748+0000 7f7e752b8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e70130b10 0x7f7e701c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:19.749 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.748+0000 7f7e752b8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e70130b10 0x7f7e701c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41690/0 (socket says 172.21.15.96:41690) 2024-03-21T06:37:19.749 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.748+0000 7f7e752b8640 1 -- 172.21.15.96:0/4036656738 learned_addr learned my addr 172.21.15.96:0/4036656738 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:19.749 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.748+0000 7f7e752b8640 1 -- 172.21.15.96:0/4036656738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7011acb0 con 0x7f7e70130b10 2024-03-21T06:37:19.749 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.748+0000 7f7e752b8640 1 --2- 172.21.15.96:0/4036656738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e70130b10 0x7f7e701c8d80 crc :-1 s=READY pgs=12919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:19.749 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.748+0000 7f7e65ffb640 1 -- 172.21.15.96:0/4036656738 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7e6c0059d0 con 0x7f7e70130b10 2024-03-21T06:37:19.749 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.748+0000 7f7e65ffb640 1 -- 172.21.15.96:0/4036656738 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7e6c005b30 con 0x7f7e70130b10 2024-03-21T06:37:19.750 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.748+0000 7f7e7751c640 1 -- 172.21.15.96:0/4036656738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e70118dc0 con 0x7f7e70130b10 2024-03-21T06:37:19.750 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.748+0000 7f7e65ffb640 1 -- 172.21.15.96:0/4036656738 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7e6c005dd0 con 0x7f7e70130b10 2024-03-21T06:37:19.750 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.749+0000 7f7e7751c640 1 -- 172.21.15.96:0/4036656738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e70119760 con 0x7f7e70130b10 2024-03-21T06:37:19.750 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.749+0000 7f7e65ffb640 1 -- 172.21.15.96:0/4036656738 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f7e6c0070b0 con 0x7f7e70130b10 2024-03-21T06:37:19.750 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.749+0000 7f7e65ffb640 1 --2- 172.21.15.96:0/4036656738 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f7e54038260 0x7f7e5403a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:19.750 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.750+0000 7f7e65ffb640 1 -- 172.21.15.96:0/4036656738 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(700..700 src has 402..700) v4 ==== 4166+0+0 (crc 0 0 0) 0x7f7e6c047e20 con 0x7f7e70130b10 2024-03-21T06:37:19.750 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.750+0000 7f7e7751c640 1 -- 172.21.15.96:0/4036656738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e70068e80 con 0x7f7e70130b10 2024-03-21T06:37:19.750 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.750+0000 7f7e74ab7640 1 --2- 172.21.15.96:0/4036656738 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f7e54038260 0x7f7e5403a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:19.750 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.750+0000 7f7e74ab7640 1 --2- 172.21.15.96:0/4036656738 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f7e54038260 0x7f7e5403a770 crc :-1 s=READY pgs=3845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:19.754 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.755+0000 7f7e65ffb640 1 -- 172.21.15.96:0/4036656738 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f7e6c00d330 con 0x7f7e70130b10 2024-03-21T06:37:19.935 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:19.936+0000 7f7e7751c640 1 -- 172.21.15.96:0/4036656738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "416ac5b8-f35e-473b-b584-9498e7bbba57", "app": "rados"} v 0) v1 -- 0x7f7e7011f410 con 0x7f7e70130b10 2024-03-21T06:37:20.464 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:20.465+0000 7f7e65ffb640 1 -- 172.21.15.96:0/4036656738 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "416ac5b8-f35e-473b-b584-9498e7bbba57", "app": "rados"}]=0 enabled application 'rados' on pool '416ac5b8-f35e-473b-b584-9498e7bbba57' v701) v1 ==== 213+0+0 (crc 0 0 0) 0x7f7e6c00d150 con 0x7f7e70130b10 2024-03-21T06:37:20.631 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:20.632+0000 7f7e7751c640 1 -- 172.21.15.96:0/4036656738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "416ac5b8-f35e-473b-b584-9498e7bbba57", "app": "rados"} v 0) v1 -- 0x7f7e700008d0 con 0x7f7e70130b10 2024-03-21T06:37:21.467 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.468+0000 7f7e65ffb640 1 -- 172.21.15.96:0/4036656738 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "416ac5b8-f35e-473b-b584-9498e7bbba57", "app": "rados"}]=0 enabled application 'rados' on pool '416ac5b8-f35e-473b-b584-9498e7bbba57' v702) v1 ==== 213+0+0 (crc 0 0 0) 0x7f7e6c00d510 con 0x7f7e70130b10 2024-03-21T06:37:21.467 INFO:tasks.workunit.client.0.smithi096.stderr:enabled application 'rados' on pool '416ac5b8-f35e-473b-b584-9498e7bbba57' 2024-03-21T06:37:21.470 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.470+0000 7f7e7751c640 1 -- 172.21.15.96:0/4036656738 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f7e54038260 msgr2=0x7f7e5403a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:21.470 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.470+0000 7f7e7751c640 1 --2- 172.21.15.96:0/4036656738 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f7e54038260 0x7f7e5403a770 crc :-1 s=READY pgs=3845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:21.470 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.470+0000 7f7e7751c640 1 -- 172.21.15.96:0/4036656738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e70130b10 msgr2=0x7f7e701c8d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:21.470 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.470+0000 7f7e7751c640 1 --2- 172.21.15.96:0/4036656738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e70130b10 0x7f7e701c8d80 crc :-1 s=READY pgs=12919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:21.470 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.471+0000 7f7e7751c640 1 -- 172.21.15.96:0/4036656738 shutdown_connections 2024-03-21T06:37:21.470 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.471+0000 7f7e7751c640 1 --2- 172.21.15.96:0/4036656738 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f7e54038260 0x7f7e5403a770 unknown :-1 s=CLOSED pgs=3845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:21.470 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.471+0000 7f7e7751c640 1 --2- 172.21.15.96:0/4036656738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e70130b10 0x7f7e701c8d80 unknown :-1 s=CLOSED pgs=12919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:21.470 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.471+0000 7f7e7751c640 1 -- 172.21.15.96:0/4036656738 >> 172.21.15.96:0/4036656738 conn(0x7f7e700a05c0 msgr2=0x7f7e7012d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:21.470 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.471+0000 7f7e7751c640 1 -- 172.21.15.96:0/4036656738 shutdown_connections 2024-03-21T06:37:21.470 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.471+0000 7f7e7751c640 1 -- 172.21.15.96:0/4036656738 wait complete. 2024-03-21T06:37:21.481 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph osd pool set-quota 416ac5b8-f35e-473b-b584-9498e7bbba57 max_objects 10 2024-03-21T06:37:21.575 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.568+0000 7f338bd5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:21.575 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.576+0000 7f338bd5c640 1 Processor -- start 2024-03-21T06:37:21.575 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.576+0000 7f338bd5c640 1 -- start start 2024-03-21T06:37:21.577 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.577+0000 7f338bd5c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3384130b10 0x7f3384130f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:21.577 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.577+0000 7f338bd5c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f338409ed40 con 0x7f338409d2c0 2024-03-21T06:37:21.577 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.577+0000 7f338bd5c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3384067f10 con 0x7f3384130b10 2024-03-21T06:37:21.577 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.577+0000 7f3389af8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3384130b10 0x7f3384130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:21.577 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.577+0000 7f33892f7640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f338409d2c0 0x7f338409d6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:48028/0 (socket says 172.21.15.96:48028) 2024-03-21T06:37:21.578 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.577+0000 7f33892f7640 1 -- 172.21.15.96:0/2244363436 learned_addr learned my addr 172.21.15.96:0/2244363436 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:37:21.578 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.578+0000 7f3389af8640 1 -- 172.21.15.96:0/2244363436 >> v1:172.21.15.96:6789/0 conn(0x7f338409d2c0 legacy=0x7f338409d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:21.578 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.578+0000 7f3389af8640 1 -- 172.21.15.96:0/2244363436 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33841306e0 con 0x7f3384130b10 2024-03-21T06:37:21.578 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.578+0000 7f3388af6640 1 -- 172.21.15.96:0/2244363436 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810708653 0 0) 0x7f338409ed40 con 0x7f338409d2c0 2024-03-21T06:37:21.578 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.578+0000 7f3389af8640 1 --2- 172.21.15.96:0/2244363436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3384130b10 0x7f3384130f20 crc :-1 s=READY pgs=12921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a6f79560f67b74 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:21.578 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.578+0000 7f3388af6640 1 -- 172.21.15.96:0/2244363436 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3380009ce0 con 0x7f3384130b10 2024-03-21T06:37:21.578 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.578+0000 7f3388af6640 1 -- 172.21.15.96:0/2244363436 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3380009e40 con 0x7f3384130b10 2024-03-21T06:37:21.578 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.578+0000 7f3388af6640 1 -- 172.21.15.96:0/2244363436 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f338000e360 con 0x7f3384130b10 2024-03-21T06:37:21.581 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.578+0000 7f338bd5c640 1 -- 172.21.15.96:0/2244363436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3384130b10 msgr2=0x7f3384130f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:21.581 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.578+0000 7f338bd5c640 1 --2- 172.21.15.96:0/2244363436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3384130b10 0x7f3384130f20 crc :-1 s=READY pgs=12921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:21.581 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.579+0000 7f338bd5c640 1 -- 172.21.15.96:0/2244363436 shutdown_connections 2024-03-21T06:37:21.581 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.579+0000 7f338bd5c640 1 --2- 172.21.15.96:0/2244363436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3384130b10 0x7f3384130f20 unknown :-1 s=CLOSED pgs=12921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:21.581 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.579+0000 7f338bd5c640 1 -- 172.21.15.96:0/2244363436 >> 172.21.15.96:0/2244363436 conn(0x7f33840a05c0 msgr2=0x7f33840a09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:21.582 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.579+0000 7f338bd5c640 1 -- 172.21.15.96:0/2244363436 shutdown_connections 2024-03-21T06:37:21.582 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.579+0000 7f338bd5c640 1 -- 172.21.15.96:0/2244363436 wait complete. 2024-03-21T06:37:21.582 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.579+0000 7f338bd5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:21.582 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.580+0000 7f338bd5c640 1 Processor -- start 2024-03-21T06:37:21.582 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.580+0000 7f338bd5c640 1 -- start start 2024-03-21T06:37:21.582 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.580+0000 7f338bd5c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f338409d2c0 0x7f338411ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:21.582 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.580+0000 7f338bd5c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3384122560 con 0x7f338409d2c0 2024-03-21T06:37:21.582 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.580+0000 7f3389af8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f338409d2c0 0x7f338411ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:21.582 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.580+0000 7f3389af8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f338409d2c0 0x7f338411ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41712/0 (socket says 172.21.15.96:41712) 2024-03-21T06:37:21.582 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.581+0000 7f3389af8640 1 -- 172.21.15.96:0/1391354453 learned_addr learned my addr 172.21.15.96:0/1391354453 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:21.582 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.581+0000 7f3389af8640 1 -- 172.21.15.96:0/1391354453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338411e8a0 con 0x7f338409d2c0 2024-03-21T06:37:21.582 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.581+0000 7f3389af8640 1 --2- 172.21.15.96:0/1391354453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f338409d2c0 0x7f338411ed60 crc :-1 s=READY pgs=12922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:21.582 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.581+0000 7f337a7fc640 1 -- 172.21.15.96:0/1391354453 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3380014620 con 0x7f338409d2c0 2024-03-21T06:37:21.583 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.581+0000 7f337a7fc640 1 -- 172.21.15.96:0/1391354453 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3380014780 con 0x7f338409d2c0 2024-03-21T06:37:21.583 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.581+0000 7f338bd5c640 1 -- 172.21.15.96:0/1391354453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338411e590 con 0x7f338409d2c0 2024-03-21T06:37:21.583 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.581+0000 7f337a7fc640 1 -- 172.21.15.96:0/1391354453 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3380014a20 con 0x7f338409d2c0 2024-03-21T06:37:21.583 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.581+0000 7f338bd5c640 1 -- 172.21.15.96:0/1391354453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3384121970 con 0x7f338409d2c0 2024-03-21T06:37:21.583 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.582+0000 7f337a7fc640 1 -- 172.21.15.96:0/1391354453 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f3380014bd0 con 0x7f338409d2c0 2024-03-21T06:37:21.583 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.582+0000 7f338bd5c640 1 -- 172.21.15.96:0/1391354453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3384068e80 con 0x7f338409d2c0 2024-03-21T06:37:21.583 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.582+0000 7f337a7fc640 1 --2- 172.21.15.96:0/1391354453 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3368038260 0x7f336803a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:21.583 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.582+0000 7f337a7fc640 1 -- 172.21.15.96:0/1391354453 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(702..702 src has 402..702) v4 ==== 4179+0+0 (crc 0 0 0) 0x7f3380019040 con 0x7f338409d2c0 2024-03-21T06:37:21.583 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.582+0000 7f33892f7640 1 --2- 172.21.15.96:0/1391354453 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3368038260 0x7f336803a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:21.584 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.583+0000 7f33892f7640 1 --2- 172.21.15.96:0/1391354453 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3368038260 0x7f336803a770 crc :-1 s=READY pgs=3846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:21.586 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.587+0000 7f337a7fc640 1 -- 172.21.15.96:0/1391354453 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f3380011dc0 con 0x7f338409d2c0 2024-03-21T06:37:21.759 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:21.760+0000 7f338bd5c640 1 -- 172.21.15.96:0/1391354453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "416ac5b8-f35e-473b-b584-9498e7bbba57", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f3384120910 con 0x7f338409d2c0 2024-03-21T06:37:22.450 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:22.451+0000 7f337a7fc640 1 -- 172.21.15.96:0/1391354453 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "416ac5b8-f35e-473b-b584-9498e7bbba57", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 416ac5b8-f35e-473b-b584-9498e7bbba57 v703) v1 ==== 223+0+0 (crc 0 0 0) 0x7f338000ea80 con 0x7f338409d2c0 2024-03-21T06:37:22.558 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:22.559+0000 7f338bd5c640 1 -- 172.21.15.96:0/1391354453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "416ac5b8-f35e-473b-b584-9498e7bbba57", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f33840008d0 con 0x7f338409d2c0 2024-03-21T06:37:23.476 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:23.477+0000 7f337a7fc640 1 -- 172.21.15.96:0/1391354453 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "416ac5b8-f35e-473b-b584-9498e7bbba57", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 416ac5b8-f35e-473b-b584-9498e7bbba57 v704) v1 ==== 223+0+0 (crc 0 0 0) 0x7f338000ec60 con 0x7f338409d2c0 2024-03-21T06:37:23.476 INFO:tasks.workunit.client.0.smithi096.stderr:set-quota max_objects = 10 for pool 416ac5b8-f35e-473b-b584-9498e7bbba57 2024-03-21T06:37:23.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:23.479+0000 7f338bd5c640 1 -- 172.21.15.96:0/1391354453 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3368038260 msgr2=0x7f336803a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:23.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:23.479+0000 7f338bd5c640 1 --2- 172.21.15.96:0/1391354453 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3368038260 0x7f336803a770 crc :-1 s=READY pgs=3846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:23.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:23.479+0000 7f338bd5c640 1 -- 172.21.15.96:0/1391354453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f338409d2c0 msgr2=0x7f338411ed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:23.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:23.479+0000 7f338bd5c640 1 --2- 172.21.15.96:0/1391354453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f338409d2c0 0x7f338411ed60 crc :-1 s=READY pgs=12922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:23.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:23.479+0000 7f338bd5c640 1 -- 172.21.15.96:0/1391354453 shutdown_connections 2024-03-21T06:37:23.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:23.479+0000 7f338bd5c640 1 --2- 172.21.15.96:0/1391354453 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3368038260 0x7f336803a770 unknown :-1 s=CLOSED pgs=3846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:23.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:23.479+0000 7f338bd5c640 1 --2- 172.21.15.96:0/1391354453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f338409d2c0 0x7f338411ed60 unknown :-1 s=CLOSED pgs=12922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:23.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:23.479+0000 7f338bd5c640 1 -- 172.21.15.96:0/1391354453 >> 172.21.15.96:0/1391354453 conn(0x7f33840a05c0 msgr2=0x7f3384071440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:23.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:23.479+0000 7f338bd5c640 1 -- 172.21.15.96:0/1391354453 shutdown_connections 2024-03-21T06:37:23.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:23.479+0000 7f338bd5c640 1 -- 172.21.15.96:0/1391354453 wait complete. 2024-03-21T06:37:23.493 INFO:tasks.workunit.client.0.smithi096.stderr:+ sleep 30 2024-03-21T06:37:53.495 INFO:tasks.workunit.client.0.smithi096.stderr:++ seq 1 10 2024-03-21T06:37:53.496 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:37:53.496 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 416ac5b8-f35e-473b-b584-9498e7bbba57 put obj1 /etc/passwd 2024-03-21T06:37:53.533 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.533+0000 7f2d68472e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.537 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.538+0000 7f2d68472e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.537 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.538+0000 7f2d68472e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.538 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.539+0000 7f2d68472e00 1 Processor -- start 2024-03-21T06:37:53.538 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.539+0000 7f2d68472e00 1 -- start start 2024-03-21T06:37:53.540 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.539+0000 7f2d68472e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b45d83de30 0x55b45d972a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.540 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.539+0000 7f2d68472e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b45d7f0ea0 con 0x55b45d83de30 2024-03-21T06:37:53.540 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.539+0000 7f2d6099a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b45d83de30 0x55b45d972a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.540 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.539+0000 7f2d6099a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b45d83de30 0x55b45d972a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43116/0 (socket says 172.21.15.96:43116) 2024-03-21T06:37:53.540 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.539+0000 7f2d6099a640 1 -- 172.21.15.96:0/886363102 learned_addr learned my addr 172.21.15.96:0/886363102 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.540 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.540+0000 7f2d6099a640 1 -- 172.21.15.96:0/886363102 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b45d793a30 con 0x55b45d83de30 2024-03-21T06:37:53.540 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.540+0000 7f2d6099a640 1 --2- 172.21.15.96:0/886363102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b45d83de30 0x55b45d972a50 crc :-1 s=READY pgs=12925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5177e062647a9e23 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.540 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.540+0000 7f2d5affd640 1 -- 172.21.15.96:0/886363102 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2d4c02fac0 con 0x55b45d83de30 2024-03-21T06:37:53.540 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.540+0000 7f2d5affd640 1 -- 172.21.15.96:0/886363102 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f2d4c02fc20 con 0x55b45d83de30 2024-03-21T06:37:53.540 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.540+0000 7f2d5affd640 1 -- 172.21.15.96:0/886363102 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2d4c02fec0 con 0x55b45d83de30 2024-03-21T06:37:53.540 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.540+0000 7f2d68472e00 1 -- 172.21.15.96:0/886363102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b45d83de30 msgr2=0x55b45d972a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.541 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.540+0000 7f2d68472e00 1 --2- 172.21.15.96:0/886363102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b45d83de30 0x55b45d972a50 crc :-1 s=READY pgs=12925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.541 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.541+0000 7f2d68472e00 1 -- 172.21.15.96:0/886363102 shutdown_connections 2024-03-21T06:37:53.541 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.541+0000 7f2d68472e00 1 --2- 172.21.15.96:0/886363102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b45d83de30 0x55b45d972a50 unknown :-1 s=CLOSED pgs=12925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.541 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.541+0000 7f2d68472e00 1 -- 172.21.15.96:0/886363102 >> 172.21.15.96:0/886363102 conn(0x55b45d765e00 msgr2=0x55b45d84fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.541 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.541+0000 7f2d68472e00 1 -- 172.21.15.96:0/886363102 shutdown_connections 2024-03-21T06:37:53.541 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.541+0000 7f2d68472e00 1 -- 172.21.15.96:0/886363102 wait complete. 2024-03-21T06:37:53.543 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.541+0000 7f2d68472e00 1 Processor -- start 2024-03-21T06:37:53.543 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.541+0000 7f2d68472e00 1 -- start start 2024-03-21T06:37:53.543 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.542+0000 7f2d68472e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b45d83de30 0x55b45d97fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.543 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.542+0000 7f2d68472e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b45d791550 con 0x55b45d83de30 2024-03-21T06:37:53.543 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.542+0000 7f2d6099a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b45d83de30 0x55b45d97fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.544 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.542+0000 7f2d6099a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b45d83de30 0x55b45d97fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43126/0 (socket says 172.21.15.96:43126) 2024-03-21T06:37:53.544 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.542+0000 7f2d6099a640 1 -- 172.21.15.96:0/1005132302 learned_addr learned my addr 172.21.15.96:0/1005132302 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.544 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.542+0000 7f2d6099a640 1 -- 172.21.15.96:0/1005132302 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b45d786680 con 0x55b45d83de30 2024-03-21T06:37:53.544 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.542+0000 7f2d6099a640 1 --2- 172.21.15.96:0/1005132302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b45d83de30 0x55b45d97fd40 crc :-1 s=READY pgs=12926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.544 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.543+0000 7f2d59ffb640 1 -- 172.21.15.96:0/1005132302 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2d4c03a040 con 0x55b45d83de30 2024-03-21T06:37:53.544 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.543+0000 7f2d59ffb640 1 -- 172.21.15.96:0/1005132302 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f2d4c02fa40 con 0x55b45d83de30 2024-03-21T06:37:53.544 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.543+0000 7f2d68472e00 1 -- 172.21.15.96:0/1005132302 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b45d7860e0 con 0x55b45d83de30 2024-03-21T06:37:53.544 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.543+0000 7f2d59ffb640 1 -- 172.21.15.96:0/1005132302 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2d4c036d30 con 0x55b45d83de30 2024-03-21T06:37:53.544 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.543+0000 7f2d68472e00 1 -- 172.21.15.96:0/1005132302 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b45d8d85d0 con 0x55b45d83de30 2024-03-21T06:37:53.544 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.543+0000 7f2d59ffb640 1 -- 172.21.15.96:0/1005132302 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f2d4c03d040 con 0x55b45d83de30 2024-03-21T06:37:53.544 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.544+0000 7f2d59ffb640 1 --2- 172.21.15.96:0/1005132302 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f2d48036b80 0x7f2d48039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.544 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.544+0000 7f2d59ffb640 1 -- 172.21.15.96:0/1005132302 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(705..705 src has 402..705) v4 ==== 4179+0+0 (crc 0 0 0) 0x7f2d4c06bd10 con 0x55b45d83de30 2024-03-21T06:37:53.544 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.544+0000 7f2d68472e00 1 --2- 172.21.15.96:0/1005132302 >> v2:172.21.15.184:6811/276480837 conn(0x55b45d8eadf0 0x55b45d980790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.544 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.544+0000 7f2d68472e00 1 -- 172.21.15.96:0/1005132302 --> v2:172.21.15.184:6811/276480837 -- osd_op(unknown.0.0:1 201.7 201:ff7b1f36:::obj1:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e705) v9 -- 0x55b45d8d5880 con 0x55b45d8eadf0 2024-03-21T06:37:53.545 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.544+0000 7f2d5bfff640 1 --2- 172.21.15.96:0/1005132302 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f2d48036b80 0x7f2d48039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.545 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.545+0000 7f2d5bfff640 1 --2- 172.21.15.96:0/1005132302 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f2d48036b80 0x7f2d48039090 crc :-1 s=READY pgs=3847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.545 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.545+0000 7f2d6119b640 1 --2- 172.21.15.96:0/1005132302 >> v2:172.21.15.184:6811/276480837 conn(0x55b45d8eadf0 0x55b45d980790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:37:53.547 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.548+0000 7f2d6119b640 1 --2- 172.21.15.96:0/1005132302 >> v2:172.21.15.184:6811/276480837 conn(0x55b45d8eadf0 0x55b45d980790 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.558 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.557+0000 7f2d6119b640 1 -- 172.21.15.96:0/1005132302 <== osd.3 v2:172.21.15.184:6811/276480837 1 ==== osd_op_reply(1 obj1 [writefull 0~11812] v705'1 uv1 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f2d5400c430 con 0x55b45d8eadf0 2024-03-21T06:37:53.558 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.557+0000 7f2d68472e00 1 -- 172.21.15.96:0/1005132302 >> v2:172.21.15.184:6811/276480837 conn(0x55b45d8eadf0 msgr2=0x55b45d980790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.558 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.557+0000 7f2d68472e00 1 --2- 172.21.15.96:0/1005132302 >> v2:172.21.15.184:6811/276480837 conn(0x55b45d8eadf0 0x55b45d980790 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.558 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.557+0000 7f2d68472e00 1 -- 172.21.15.96:0/1005132302 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f2d48036b80 msgr2=0x7f2d48039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.558 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.557+0000 7f2d68472e00 1 --2- 172.21.15.96:0/1005132302 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f2d48036b80 0x7f2d48039090 crc :-1 s=READY pgs=3847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.559 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.557+0000 7f2d68472e00 1 -- 172.21.15.96:0/1005132302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b45d83de30 msgr2=0x55b45d97fd40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.559 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.557+0000 7f2d68472e00 1 --2- 172.21.15.96:0/1005132302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b45d83de30 0x55b45d97fd40 crc :-1 s=READY pgs=12926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.559 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.557+0000 7f2d68472e00 1 -- 172.21.15.96:0/1005132302 shutdown_connections 2024-03-21T06:37:53.559 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.557+0000 7f2d68472e00 1 --2- 172.21.15.96:0/1005132302 >> v2:172.21.15.184:6811/276480837 conn(0x55b45d8eadf0 0x55b45d980790 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.559 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.557+0000 7f2d68472e00 1 --2- 172.21.15.96:0/1005132302 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f2d48036b80 0x7f2d48039090 unknown :-1 s=CLOSED pgs=3847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.559 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.557+0000 7f2d68472e00 1 --2- 172.21.15.96:0/1005132302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b45d83de30 0x55b45d97fd40 unknown :-1 s=CLOSED pgs=12926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.559 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.557+0000 7f2d68472e00 1 -- 172.21.15.96:0/1005132302 >> 172.21.15.96:0/1005132302 conn(0x55b45d765e00 msgr2=0x55b45d84fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.559 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.557+0000 7f2d68472e00 1 -- 172.21.15.96:0/1005132302 shutdown_connections 2024-03-21T06:37:53.559 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.558+0000 7f2d68472e00 1 -- 172.21.15.96:0/1005132302 wait complete. 2024-03-21T06:37:53.560 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:37:53.560 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 416ac5b8-f35e-473b-b584-9498e7bbba57 put obj2 /etc/passwd 2024-03-21T06:37:53.591 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.592+0000 7f4600870e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.596 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.597+0000 7f4600870e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.596 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.597+0000 7f4600870e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.596 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.597+0000 7f4600870e00 1 Processor -- start 2024-03-21T06:37:53.596 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.597+0000 7f4600870e00 1 -- start start 2024-03-21T06:37:53.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.598+0000 7f4600870e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56101355cdf0 0x5610135e9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.598+0000 7f4600870e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x561013462ea0 con 0x56101355cdf0 2024-03-21T06:37:53.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.598+0000 7f45f8d9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56101355cdf0 0x5610135e9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.598+0000 7f45f8d9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56101355cdf0 0x5610135e9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43152/0 (socket says 172.21.15.96:43152) 2024-03-21T06:37:53.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.598+0000 7f45f8d9a640 1 -- 172.21.15.96:0/1353222267 learned_addr learned my addr 172.21.15.96:0/1353222267 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.598+0000 7f45f8d9a640 1 -- 172.21.15.96:0/1353222267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561013405a30 con 0x56101355cdf0 2024-03-21T06:37:53.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.598+0000 7f45f8d9a640 1 --2- 172.21.15.96:0/1353222267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56101355cdf0 0x5610135e9530 crc :-1 s=READY pgs=12929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edda9e7da063cf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.598+0000 7f45f2ffd640 1 -- 172.21.15.96:0/1353222267 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f45e800c490 con 0x56101355cdf0 2024-03-21T06:37:53.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.598+0000 7f45f2ffd640 1 -- 172.21.15.96:0/1353222267 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f45e800c5f0 con 0x56101355cdf0 2024-03-21T06:37:53.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.598+0000 7f45f2ffd640 1 -- 172.21.15.96:0/1353222267 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f45e800c890 con 0x56101355cdf0 2024-03-21T06:37:53.602 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.599+0000 7f4600870e00 1 -- 172.21.15.96:0/1353222267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56101355cdf0 msgr2=0x5610135e9530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.602 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.599+0000 7f4600870e00 1 --2- 172.21.15.96:0/1353222267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56101355cdf0 0x5610135e9530 crc :-1 s=READY pgs=12929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.602 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.599+0000 7f4600870e00 1 -- 172.21.15.96:0/1353222267 shutdown_connections 2024-03-21T06:37:53.603 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.599+0000 7f4600870e00 1 --2- 172.21.15.96:0/1353222267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56101355cdf0 0x5610135e9530 unknown :-1 s=CLOSED pgs=12929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.603 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.599+0000 7f4600870e00 1 -- 172.21.15.96:0/1353222267 >> 172.21.15.96:0/1353222267 conn(0x5610133d7e00 msgr2=0x5610134c4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.603 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.599+0000 7f4600870e00 1 -- 172.21.15.96:0/1353222267 shutdown_connections 2024-03-21T06:37:53.603 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.599+0000 7f4600870e00 1 -- 172.21.15.96:0/1353222267 wait complete. 2024-03-21T06:37:53.603 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.600+0000 7f4600870e00 1 Processor -- start 2024-03-21T06:37:53.603 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.600+0000 7f4600870e00 1 -- start start 2024-03-21T06:37:53.603 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.600+0000 7f4600870e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56101355cdf0 0x5610135f1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.603 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.600+0000 7f4600870e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x561013403550 con 0x56101355cdf0 2024-03-21T06:37:53.603 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.600+0000 7f45f8d9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56101355cdf0 0x5610135f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.603 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.600+0000 7f45f8d9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56101355cdf0 0x5610135f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43168/0 (socket says 172.21.15.96:43168) 2024-03-21T06:37:53.604 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.600+0000 7f45f8d9a640 1 -- 172.21.15.96:0/2700633312 learned_addr learned my addr 172.21.15.96:0/2700633312 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.604 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.601+0000 7f45f8d9a640 1 -- 172.21.15.96:0/2700633312 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610133f8680 con 0x56101355cdf0 2024-03-21T06:37:53.604 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.601+0000 7f45f8d9a640 1 --2- 172.21.15.96:0/2700633312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56101355cdf0 0x5610135f1750 crc :-1 s=READY pgs=12930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.604 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.601+0000 7f45f1ffb640 1 -- 172.21.15.96:0/2700633312 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f45e80111e0 con 0x56101355cdf0 2024-03-21T06:37:53.604 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.601+0000 7f45f1ffb640 1 -- 172.21.15.96:0/2700633312 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f45e8011980 con 0x56101355cdf0 2024-03-21T06:37:53.604 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.601+0000 7f45f1ffb640 1 -- 172.21.15.96:0/2700633312 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f45e8011af0 con 0x56101355cdf0 2024-03-21T06:37:53.604 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.601+0000 7f4600870e00 1 -- 172.21.15.96:0/2700633312 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5610133f80e0 con 0x56101355cdf0 2024-03-21T06:37:53.604 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.601+0000 7f4600870e00 1 -- 172.21.15.96:0/2700633312 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610135e62f0 con 0x56101355cdf0 2024-03-21T06:37:53.604 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.602+0000 7f45f1ffb640 1 -- 172.21.15.96:0/2700633312 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f45e8011ca0 con 0x56101355cdf0 2024-03-21T06:37:53.604 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.602+0000 7f45f1ffb640 1 --2- 172.21.15.96:0/2700633312 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f45d8036b80 0x7f45d8039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.604 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.602+0000 7f45f1ffb640 1 -- 172.21.15.96:0/2700633312 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(705..705 src has 402..705) v4 ==== 4179+0+0 (crc 0 0 0) 0x7f45e8048500 con 0x56101355cdf0 2024-03-21T06:37:53.604 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.602+0000 7f4600870e00 1 --2- 172.21.15.96:0/2700633312 >> v2:172.21.15.184:6804/155456265 conn(0x56101355fd40 0x56101355b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.604 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.602+0000 7f4600870e00 1 -- 172.21.15.96:0/2700633312 --> v2:172.21.15.184:6804/155456265 -- osd_op(unknown.0.0:1 201.8 201:104778fc:::obj2:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e705) v9 -- 0x561013546210 con 0x56101355fd40 2024-03-21T06:37:53.604 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.603+0000 7f45f3fff640 1 --2- 172.21.15.96:0/2700633312 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f45d8036b80 0x7f45d8039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.605 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.603+0000 7f45f959b640 1 --2- 172.21.15.96:0/2700633312 >> v2:172.21.15.184:6804/155456265 conn(0x56101355fd40 0x56101355b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:37:53.605 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.603+0000 7f45f3fff640 1 --2- 172.21.15.96:0/2700633312 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f45d8036b80 0x7f45d8039090 crc :-1 s=READY pgs=3848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.605 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.605+0000 7f45f959b640 1 --2- 172.21.15.96:0/2700633312 >> v2:172.21.15.184:6804/155456265 conn(0x56101355fd40 0x56101355b520 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.614 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.614+0000 7f45f959b640 1 -- 172.21.15.96:0/2700633312 <== osd.2 v2:172.21.15.184:6804/155456265 1 ==== osd_op_reply(1 obj2 [writefull 0~11812] v705'1 uv1 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f45ec00a430 con 0x56101355fd40 2024-03-21T06:37:53.614 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.614+0000 7f4600870e00 1 -- 172.21.15.96:0/2700633312 >> v2:172.21.15.184:6804/155456265 conn(0x56101355fd40 msgr2=0x56101355b520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.614 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.614+0000 7f4600870e00 1 --2- 172.21.15.96:0/2700633312 >> v2:172.21.15.184:6804/155456265 conn(0x56101355fd40 0x56101355b520 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.614 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.614+0000 7f4600870e00 1 -- 172.21.15.96:0/2700633312 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f45d8036b80 msgr2=0x7f45d8039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.614 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.614+0000 7f4600870e00 1 --2- 172.21.15.96:0/2700633312 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f45d8036b80 0x7f45d8039090 crc :-1 s=READY pgs=3848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.614 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.614+0000 7f4600870e00 1 -- 172.21.15.96:0/2700633312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56101355cdf0 msgr2=0x5610135f1750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.614 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.614+0000 7f4600870e00 1 --2- 172.21.15.96:0/2700633312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56101355cdf0 0x5610135f1750 crc :-1 s=READY pgs=12930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.614 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.614+0000 7f4600870e00 1 -- 172.21.15.96:0/2700633312 shutdown_connections 2024-03-21T06:37:53.615 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.614+0000 7f4600870e00 1 --2- 172.21.15.96:0/2700633312 >> v2:172.21.15.184:6804/155456265 conn(0x56101355fd40 0x56101355b520 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.615 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.614+0000 7f4600870e00 1 --2- 172.21.15.96:0/2700633312 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f45d8036b80 0x7f45d8039090 unknown :-1 s=CLOSED pgs=3848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.615 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.614+0000 7f4600870e00 1 --2- 172.21.15.96:0/2700633312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56101355cdf0 0x5610135f1750 unknown :-1 s=CLOSED pgs=12930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.615 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.614+0000 7f4600870e00 1 -- 172.21.15.96:0/2700633312 >> 172.21.15.96:0/2700633312 conn(0x5610133d7e00 msgr2=0x5610134c4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.615 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.615+0000 7f4600870e00 1 -- 172.21.15.96:0/2700633312 shutdown_connections 2024-03-21T06:37:53.615 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.615+0000 7f4600870e00 1 -- 172.21.15.96:0/2700633312 wait complete. 2024-03-21T06:37:53.616 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:37:53.616 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 416ac5b8-f35e-473b-b584-9498e7bbba57 put obj3 /etc/passwd 2024-03-21T06:37:53.646 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.646+0000 7f5e79c70e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.651+0000 7f5e79c70e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.651+0000 7f5e79c70e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.651+0000 7f5e79c70e00 1 Processor -- start 2024-03-21T06:37:53.650 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.651+0000 7f5e79c70e00 1 -- start start 2024-03-21T06:37:53.652 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.652+0000 7f5e79c70e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0abc3b040 0x55d0abcba850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.652+0000 7f5e79c70e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0abbd7ea0 con 0x55d0abc3b040 2024-03-21T06:37:53.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.652+0000 7f5e7219a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0abc3b040 0x55d0abcba850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.652+0000 7f5e7219a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0abc3b040 0x55d0abcba850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43196/0 (socket says 172.21.15.96:43196) 2024-03-21T06:37:53.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.652+0000 7f5e7219a640 1 -- 172.21.15.96:0/2355647069 learned_addr learned my addr 172.21.15.96:0/2355647069 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.652+0000 7f5e7219a640 1 -- 172.21.15.96:0/2355647069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0abb7aa30 con 0x55d0abc3b040 2024-03-21T06:37:53.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.652+0000 7f5e7219a640 1 --2- 172.21.15.96:0/2355647069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0abc3b040 0x55d0abcba850 crc :-1 s=READY pgs=12933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17dd24270fa3619 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.652+0000 7f5e70997640 1 -- 172.21.15.96:0/2355647069 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5e6800c490 con 0x55d0abc3b040 2024-03-21T06:37:53.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.652+0000 7f5e70997640 1 -- 172.21.15.96:0/2355647069 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5e6800c5f0 con 0x55d0abc3b040 2024-03-21T06:37:53.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.652+0000 7f5e70997640 1 -- 172.21.15.96:0/2355647069 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5e6800c890 con 0x55d0abc3b040 2024-03-21T06:37:53.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.653+0000 7f5e79c70e00 1 -- 172.21.15.96:0/2355647069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0abc3b040 msgr2=0x55d0abcba850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.653+0000 7f5e79c70e00 1 --2- 172.21.15.96:0/2355647069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0abc3b040 0x55d0abcba850 crc :-1 s=READY pgs=12933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.653+0000 7f5e79c70e00 1 -- 172.21.15.96:0/2355647069 shutdown_connections 2024-03-21T06:37:53.653 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.653+0000 7f5e79c70e00 1 --2- 172.21.15.96:0/2355647069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0abc3b040 0x55d0abcba850 unknown :-1 s=CLOSED pgs=12933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.653+0000 7f5e79c70e00 1 -- 172.21.15.96:0/2355647069 >> 172.21.15.96:0/2355647069 conn(0x55d0abb4ce00 msgr2=0x55d0abc42000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.653+0000 7f5e79c70e00 1 -- 172.21.15.96:0/2355647069 shutdown_connections 2024-03-21T06:37:53.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.653+0000 7f5e79c70e00 1 -- 172.21.15.96:0/2355647069 wait complete. 2024-03-21T06:37:53.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.654+0000 7f5e79c70e00 1 Processor -- start 2024-03-21T06:37:53.654 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.654+0000 7f5e79c70e00 1 -- start start 2024-03-21T06:37:53.656 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.654+0000 7f5e79c70e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0abc3b040 0x55d0abcbda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.656 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.654+0000 7f5e79c70e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0abb78550 con 0x55d0abc3b040 2024-03-21T06:37:53.656 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.655+0000 7f5e7219a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0abc3b040 0x55d0abcbda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.656 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.655+0000 7f5e7219a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0abc3b040 0x55d0abcbda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43198/0 (socket says 172.21.15.96:43198) 2024-03-21T06:37:53.656 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.655+0000 7f5e7219a640 1 -- 172.21.15.96:0/607125200 learned_addr learned my addr 172.21.15.96:0/607125200 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.656 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.655+0000 7f5e7219a640 1 -- 172.21.15.96:0/607125200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0abb6d680 con 0x55d0abc3b040 2024-03-21T06:37:53.657 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.655+0000 7f5e7219a640 1 --2- 172.21.15.96:0/607125200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0abc3b040 0x55d0abcbda50 crc :-1 s=READY pgs=12934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.657 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.655+0000 7f5e637fe640 1 -- 172.21.15.96:0/607125200 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5e680111e0 con 0x55d0abc3b040 2024-03-21T06:37:53.657 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.655+0000 7f5e637fe640 1 -- 172.21.15.96:0/607125200 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5e680117a0 con 0x55d0abc3b040 2024-03-21T06:37:53.657 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.655+0000 7f5e79c70e00 1 -- 172.21.15.96:0/607125200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d0abb6d0e0 con 0x55d0abc3b040 2024-03-21T06:37:53.657 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.655+0000 7f5e637fe640 1 -- 172.21.15.96:0/607125200 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5e68011910 con 0x55d0abc3b040 2024-03-21T06:37:53.657 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.655+0000 7f5e79c70e00 1 -- 172.21.15.96:0/607125200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0abcc0ed0 con 0x55d0abc3b040 2024-03-21T06:37:53.657 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.656+0000 7f5e637fe640 1 -- 172.21.15.96:0/607125200 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f5e68011ac0 con 0x55d0abc3b040 2024-03-21T06:37:53.657 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.656+0000 7f5e637fe640 1 --2- 172.21.15.96:0/607125200 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5e50036b80 0x7f5e50039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.657 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.656+0000 7f5e637fe640 1 -- 172.21.15.96:0/607125200 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(705..705 src has 402..705) v4 ==== 4179+0+0 (crc 0 0 0) 0x7f5e68048430 con 0x55d0abc3b040 2024-03-21T06:37:53.657 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.656+0000 7f5e71999640 1 --2- 172.21.15.96:0/607125200 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5e50036b80 0x7f5e50039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.657 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.657+0000 7f5e79c70e00 1 --2- 172.21.15.96:0/607125200 >> v2:172.21.15.96:6805/13988474 conn(0x55d0abcd10c0 0x55d0abd64a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.657 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.657+0000 7f5e79c70e00 1 -- 172.21.15.96:0/607125200 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:1 201.1 201:8dd16f86:::obj3:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e705) v9 -- 0x55d0abcc1650 con 0x55d0abcd10c0 2024-03-21T06:37:53.657 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.657+0000 7f5e71999640 1 --2- 172.21.15.96:0/607125200 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5e50036b80 0x7f5e50039090 crc :-1 s=READY pgs=3849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.657 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.657+0000 7f5e7299b640 1 --2- 172.21.15.96:0/607125200 >> v2:172.21.15.96:6805/13988474 conn(0x55d0abcd10c0 0x55d0abd64a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:37:53.658 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.659+0000 7f5e7299b640 1 --2- 172.21.15.96:0/607125200 >> v2:172.21.15.96:6805/13988474 conn(0x55d0abcd10c0 0x55d0abd64a30 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.667 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.667+0000 7f5e7299b640 1 -- 172.21.15.96:0/607125200 <== osd.1 v2:172.21.15.96:6805/13988474 1 ==== osd_op_reply(1 obj3 [writefull 0~11812] v705'1 uv1 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f5e6400a430 con 0x55d0abcd10c0 2024-03-21T06:37:53.667 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.667+0000 7f5e79c70e00 1 -- 172.21.15.96:0/607125200 >> v2:172.21.15.96:6805/13988474 conn(0x55d0abcd10c0 msgr2=0x55d0abd64a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.667 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.667+0000 7f5e79c70e00 1 --2- 172.21.15.96:0/607125200 >> v2:172.21.15.96:6805/13988474 conn(0x55d0abcd10c0 0x55d0abd64a30 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.667 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.667+0000 7f5e79c70e00 1 -- 172.21.15.96:0/607125200 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5e50036b80 msgr2=0x7f5e50039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.667 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.667+0000 7f5e79c70e00 1 --2- 172.21.15.96:0/607125200 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5e50036b80 0x7f5e50039090 crc :-1 s=READY pgs=3849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.667 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.667+0000 7f5e79c70e00 1 -- 172.21.15.96:0/607125200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0abc3b040 msgr2=0x55d0abcbda50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.667 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.667+0000 7f5e79c70e00 1 --2- 172.21.15.96:0/607125200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0abc3b040 0x55d0abcbda50 crc :-1 s=READY pgs=12934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.668 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.667+0000 7f5e79c70e00 1 -- 172.21.15.96:0/607125200 shutdown_connections 2024-03-21T06:37:53.668 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.667+0000 7f5e79c70e00 1 --2- 172.21.15.96:0/607125200 >> v2:172.21.15.96:6805/13988474 conn(0x55d0abcd10c0 0x55d0abd64a30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.668 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.667+0000 7f5e79c70e00 1 --2- 172.21.15.96:0/607125200 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5e50036b80 0x7f5e50039090 unknown :-1 s=CLOSED pgs=3849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.668 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.667+0000 7f5e79c70e00 1 --2- 172.21.15.96:0/607125200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d0abc3b040 0x55d0abcbda50 unknown :-1 s=CLOSED pgs=12934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.668 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.667+0000 7f5e79c70e00 1 -- 172.21.15.96:0/607125200 >> 172.21.15.96:0/607125200 conn(0x55d0abb4ce00 msgr2=0x55d0abc42000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.668 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.668+0000 7f5e79c70e00 1 -- 172.21.15.96:0/607125200 shutdown_connections 2024-03-21T06:37:53.668 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.668+0000 7f5e79c70e00 1 -- 172.21.15.96:0/607125200 wait complete. 2024-03-21T06:37:53.669 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:37:53.669 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 416ac5b8-f35e-473b-b584-9498e7bbba57 put obj4 /etc/passwd 2024-03-21T06:37:53.698 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.699+0000 7fc588a84e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.703 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.704+0000 7fc588a84e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.703 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.704+0000 7fc588a84e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.703 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.704+0000 7fc588a84e00 1 Processor -- start 2024-03-21T06:37:53.703 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.704+0000 7fc588a84e00 1 -- start start 2024-03-21T06:37:53.705 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.705+0000 7fc588a84e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586cfee8e30 0x5586d001da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.705 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.705+0000 7fc588a84e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cfe9bea0 con 0x5586cfee8e30 2024-03-21T06:37:53.705 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.705+0000 7fc581b9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586cfee8e30 0x5586d001da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.705 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.705+0000 7fc581b9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586cfee8e30 0x5586d001da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43228/0 (socket says 172.21.15.96:43228) 2024-03-21T06:37:53.705 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.705+0000 7fc581b9a640 1 -- 172.21.15.96:0/364723195 learned_addr learned my addr 172.21.15.96:0/364723195 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.705 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.706+0000 7fc581b9a640 1 -- 172.21.15.96:0/364723195 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586cfe3ea30 con 0x5586cfee8e30 2024-03-21T06:37:53.705 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.706+0000 7fc581b9a640 1 --2- 172.21.15.96:0/364723195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586cfee8e30 0x5586d001da50 crc :-1 s=READY pgs=12937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a808046eac988cfd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.706 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.706+0000 7fc573fff640 1 -- 172.21.15.96:0/364723195 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc578033040 con 0x5586cfee8e30 2024-03-21T06:37:53.706 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.706+0000 7fc573fff640 1 -- 172.21.15.96:0/364723195 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc578034040 con 0x5586cfee8e30 2024-03-21T06:37:53.706 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.706+0000 7fc573fff640 1 -- 172.21.15.96:0/364723195 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc57800a2a0 con 0x5586cfee8e30 2024-03-21T06:37:53.708 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.707+0000 7fc588a84e00 1 -- 172.21.15.96:0/364723195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586cfee8e30 msgr2=0x5586d001da50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.708 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.707+0000 7fc588a84e00 1 --2- 172.21.15.96:0/364723195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586cfee8e30 0x5586d001da50 crc :-1 s=READY pgs=12937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.708 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.707+0000 7fc588a84e00 1 -- 172.21.15.96:0/364723195 shutdown_connections 2024-03-21T06:37:53.708 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.707+0000 7fc588a84e00 1 --2- 172.21.15.96:0/364723195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586cfee8e30 0x5586d001da50 unknown :-1 s=CLOSED pgs=12937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.708 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.707+0000 7fc588a84e00 1 -- 172.21.15.96:0/364723195 >> 172.21.15.96:0/364723195 conn(0x5586cfe10e00 msgr2=0x5586cfefad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.708 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.707+0000 7fc588a84e00 1 -- 172.21.15.96:0/364723195 shutdown_connections 2024-03-21T06:37:53.708 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.707+0000 7fc588a84e00 1 -- 172.21.15.96:0/364723195 wait complete. 2024-03-21T06:37:53.709 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.707+0000 7fc588a84e00 1 Processor -- start 2024-03-21T06:37:53.709 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.707+0000 7fc588a84e00 1 -- start start 2024-03-21T06:37:53.709 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.708+0000 7fc588a84e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586cfee8e30 0x5586cff80b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.709 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.708+0000 7fc588a84e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cfe3c550 con 0x5586cfee8e30 2024-03-21T06:37:53.709 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.708+0000 7fc581b9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586cfee8e30 0x5586cff80b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.709 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.708+0000 7fc581b9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586cfee8e30 0x5586cff80b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43242/0 (socket says 172.21.15.96:43242) 2024-03-21T06:37:53.709 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.708+0000 7fc581b9a640 1 -- 172.21.15.96:0/4253606079 learned_addr learned my addr 172.21.15.96:0/4253606079 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.709 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.708+0000 7fc581b9a640 1 -- 172.21.15.96:0/4253606079 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586cfe31680 con 0x5586cfee8e30 2024-03-21T06:37:53.709 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.708+0000 7fc581b9a640 1 --2- 172.21.15.96:0/4253606079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586cfee8e30 0x5586cff80b00 crc :-1 s=READY pgs=12938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.709 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.708+0000 7fc572ffd640 1 -- 172.21.15.96:0/4253606079 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc578033040 con 0x5586cfee8e30 2024-03-21T06:37:53.709 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.708+0000 7fc572ffd640 1 -- 172.21.15.96:0/4253606079 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc578009620 con 0x5586cfee8e30 2024-03-21T06:37:53.709 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.708+0000 7fc588a84e00 1 -- 172.21.15.96:0/4253606079 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5586cfe310e0 con 0x5586cfee8e30 2024-03-21T06:37:53.709 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.708+0000 7fc572ffd640 1 -- 172.21.15.96:0/4253606079 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc5780098c0 con 0x5586cfee8e30 2024-03-21T06:37:53.709 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.709+0000 7fc588a84e00 1 -- 172.21.15.96:0/4253606079 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586cff83ed0 con 0x5586cfee8e30 2024-03-21T06:37:53.709 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.709+0000 7fc572ffd640 1 -- 172.21.15.96:0/4253606079 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fc578009a20 con 0x5586cfee8e30 2024-03-21T06:37:53.711 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.709+0000 7fc572ffd640 1 --2- 172.21.15.96:0/4253606079 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc560036b80 0x7fc560039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.711 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.709+0000 7fc572ffd640 1 -- 172.21.15.96:0/4253606079 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(705..705 src has 402..705) v4 ==== 4179+0+0 (crc 0 0 0) 0x7fc57806b150 con 0x5586cfee8e30 2024-03-21T06:37:53.711 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.710+0000 7fc588a84e00 1 --2- 172.21.15.96:0/4253606079 >> v2:172.21.15.184:6811/276480837 conn(0x5586cff95df0 0x5586d002ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.711 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.710+0000 7fc588a84e00 1 -- 172.21.15.96:0/4253606079 --> v2:172.21.15.184:6811/276480837 -- osd_op(unknown.0.0:1 201.0 201:0ee9ae15:::obj4:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e705) v9 -- 0x5586cff84500 con 0x5586cff95df0 2024-03-21T06:37:53.711 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.710+0000 7fc581399640 1 --2- 172.21.15.96:0/4253606079 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc560036b80 0x7fc560039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.712 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.710+0000 7fc58239b640 1 --2- 172.21.15.96:0/4253606079 >> v2:172.21.15.184:6811/276480837 conn(0x5586cff95df0 0x5586d002ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:37:53.712 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.711+0000 7fc581399640 1 --2- 172.21.15.96:0/4253606079 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc560036b80 0x7fc560039090 crc :-1 s=READY pgs=3850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.713 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.713+0000 7fc58239b640 1 --2- 172.21.15.96:0/4253606079 >> v2:172.21.15.184:6811/276480837 conn(0x5586cff95df0 0x5586d002ad40 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.722 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.722+0000 7fc58239b640 1 -- 172.21.15.96:0/4253606079 <== osd.3 v2:172.21.15.184:6811/276480837 1 ==== osd_op_reply(1 obj4 [writefull 0~11812] v705'1 uv1 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fc57400a430 con 0x5586cff95df0 2024-03-21T06:37:53.722 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.722+0000 7fc588a84e00 1 -- 172.21.15.96:0/4253606079 >> v2:172.21.15.184:6811/276480837 conn(0x5586cff95df0 msgr2=0x5586d002ad40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.722 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.722+0000 7fc588a84e00 1 --2- 172.21.15.96:0/4253606079 >> v2:172.21.15.184:6811/276480837 conn(0x5586cff95df0 0x5586d002ad40 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.722 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.722+0000 7fc588a84e00 1 -- 172.21.15.96:0/4253606079 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc560036b80 msgr2=0x7fc560039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.722 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.722+0000 7fc588a84e00 1 --2- 172.21.15.96:0/4253606079 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc560036b80 0x7fc560039090 crc :-1 s=READY pgs=3850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.722 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.722+0000 7fc588a84e00 1 -- 172.21.15.96:0/4253606079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586cfee8e30 msgr2=0x5586cff80b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.723 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.722+0000 7fc588a84e00 1 --2- 172.21.15.96:0/4253606079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586cfee8e30 0x5586cff80b00 crc :-1 s=READY pgs=12938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.723 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.722+0000 7fc588a84e00 1 -- 172.21.15.96:0/4253606079 shutdown_connections 2024-03-21T06:37:53.723 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.722+0000 7fc588a84e00 1 --2- 172.21.15.96:0/4253606079 >> v2:172.21.15.184:6811/276480837 conn(0x5586cff95df0 0x5586d002ad40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.723 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.722+0000 7fc588a84e00 1 --2- 172.21.15.96:0/4253606079 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc560036b80 0x7fc560039090 unknown :-1 s=CLOSED pgs=3850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.723 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.722+0000 7fc588a84e00 1 --2- 172.21.15.96:0/4253606079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5586cfee8e30 0x5586cff80b00 unknown :-1 s=CLOSED pgs=12938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.723 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.722+0000 7fc588a84e00 1 -- 172.21.15.96:0/4253606079 >> 172.21.15.96:0/4253606079 conn(0x5586cfe10e00 msgr2=0x5586cfefad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.723 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.722+0000 7fc588a84e00 1 -- 172.21.15.96:0/4253606079 shutdown_connections 2024-03-21T06:37:53.723 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.723+0000 7fc588a84e00 1 -- 172.21.15.96:0/4253606079 wait complete. 2024-03-21T06:37:53.723 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:37:53.724 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 416ac5b8-f35e-473b-b584-9498e7bbba57 put obj5 /etc/passwd 2024-03-21T06:37:53.753 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.754+0000 7f0613c57e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.757 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.758+0000 7f0613c57e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.757 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.758+0000 7f0613c57e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.758 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.759+0000 7f0613c57e00 1 Processor -- start 2024-03-21T06:37:53.758 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.759+0000 7f0613c57e00 1 -- start start 2024-03-21T06:37:53.760 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.759+0000 7f0613c57e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645fa452640 0x5645fa458c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.760 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.759+0000 7f0613c57e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fa3ecea0 con 0x5645fa452640 2024-03-21T06:37:53.760 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.759+0000 7f060cd9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645fa452640 0x5645fa458c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.760 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.759+0000 7f060cd9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645fa452640 0x5645fa458c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43256/0 (socket says 172.21.15.96:43256) 2024-03-21T06:37:53.760 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.759+0000 7f060cd9a640 1 -- 172.21.15.96:0/1799764537 learned_addr learned my addr 172.21.15.96:0/1799764537 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.760 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.760+0000 7f060cd9a640 1 -- 172.21.15.96:0/1799764537 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645fa38fa30 con 0x5645fa452640 2024-03-21T06:37:53.760 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.760+0000 7f060cd9a640 1 --2- 172.21.15.96:0/1799764537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645fa452640 0x5645fa458c70 crc :-1 s=READY pgs=12941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344a6bdb32b538e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.761 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.760+0000 7f0606ffd640 1 -- 172.21.15.96:0/1799764537 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f060000b5d0 con 0x5645fa452640 2024-03-21T06:37:53.761 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.760+0000 7f0606ffd640 1 -- 172.21.15.96:0/1799764537 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f060000b730 con 0x5645fa452640 2024-03-21T06:37:53.761 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.760+0000 7f0606ffd640 1 -- 172.21.15.96:0/1799764537 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f060000b9d0 con 0x5645fa452640 2024-03-21T06:37:53.761 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.760+0000 7f0613c57e00 1 -- 172.21.15.96:0/1799764537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645fa452640 msgr2=0x5645fa458c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.761 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.760+0000 7f0613c57e00 1 --2- 172.21.15.96:0/1799764537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645fa452640 0x5645fa458c70 crc :-1 s=READY pgs=12941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.761 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.760+0000 7f0613c57e00 1 -- 172.21.15.96:0/1799764537 shutdown_connections 2024-03-21T06:37:53.761 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.760+0000 7f0613c57e00 1 --2- 172.21.15.96:0/1799764537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645fa452640 0x5645fa458c70 unknown :-1 s=CLOSED pgs=12941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.761 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.760+0000 7f0613c57e00 1 -- 172.21.15.96:0/1799764537 >> 172.21.15.96:0/1799764537 conn(0x5645fa361e00 msgr2=0x5645fa44e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.761 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.761+0000 7f0613c57e00 1 -- 172.21.15.96:0/1799764537 shutdown_connections 2024-03-21T06:37:53.761 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.761+0000 7f0613c57e00 1 -- 172.21.15.96:0/1799764537 wait complete. 2024-03-21T06:37:53.761 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.761+0000 7f0613c57e00 1 Processor -- start 2024-03-21T06:37:53.761 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.761+0000 7f0613c57e00 1 -- start start 2024-03-21T06:37:53.763 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.762+0000 7f0613c57e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645fa452640 0x5645fa4d4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.764 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.762+0000 7f0613c57e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645fa38d550 con 0x5645fa452640 2024-03-21T06:37:53.764 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.762+0000 7f060cd9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645fa452640 0x5645fa4d4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.764 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.762+0000 7f060cd9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645fa452640 0x5645fa4d4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43258/0 (socket says 172.21.15.96:43258) 2024-03-21T06:37:53.764 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.762+0000 7f060cd9a640 1 -- 172.21.15.96:0/1375487235 learned_addr learned my addr 172.21.15.96:0/1375487235 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.764 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.762+0000 7f060cd9a640 1 -- 172.21.15.96:0/1375487235 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645fa382680 con 0x5645fa452640 2024-03-21T06:37:53.764 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.762+0000 7f060cd9a640 1 --2- 172.21.15.96:0/1375487235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645fa452640 0x5645fa4d4d80 crc :-1 s=READY pgs=12942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.764 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.762+0000 7f0605ffb640 1 -- 172.21.15.96:0/1375487235 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0600012940 con 0x5645fa452640 2024-03-21T06:37:53.764 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.762+0000 7f0605ffb640 1 -- 172.21.15.96:0/1375487235 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f0600005040 con 0x5645fa452640 2024-03-21T06:37:53.764 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.762+0000 7f0613c57e00 1 -- 172.21.15.96:0/1375487235 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5645fa3820e0 con 0x5645fa452640 2024-03-21T06:37:53.764 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.762+0000 7f0605ffb640 1 -- 172.21.15.96:0/1375487235 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f0600012bd0 con 0x5645fa452640 2024-03-21T06:37:53.764 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.762+0000 7f0613c57e00 1 -- 172.21.15.96:0/1375487235 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645fa4d7570 con 0x5645fa452640 2024-03-21T06:37:53.764 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.763+0000 7f0605ffb640 1 -- 172.21.15.96:0/1375487235 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f060000d040 con 0x5645fa452640 2024-03-21T06:37:53.764 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.763+0000 7f0605ffb640 1 --2- 172.21.15.96:0/1375487235 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f05f4036b80 0x7f05f4039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.765 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.763+0000 7f0605ffb640 1 -- 172.21.15.96:0/1375487235 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(705..705 src has 402..705) v4 ==== 4179+0+0 (crc 0 0 0) 0x7f060000a040 con 0x5645fa452640 2024-03-21T06:37:53.765 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.764+0000 7f0613c57e00 1 --2- 172.21.15.96:0/1375487235 >> v2:172.21.15.96:6801/932229279 conn(0x5645fa455cb0 0x5645fa4e56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.765 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.764+0000 7f0613c57e00 1 -- 172.21.15.96:0/1375487235 --> v2:172.21.15.96:6801/932229279 -- osd_op(unknown.0.0:1 201.3 201:c52c9666:::obj5:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e705) v9 -- 0x5645fa4d7bf0 con 0x5645fa455cb0 2024-03-21T06:37:53.765 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.764+0000 7f0607fff640 1 --2- 172.21.15.96:0/1375487235 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f05f4036b80 0x7f05f4039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.765 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.764+0000 7f0607fff640 1 --2- 172.21.15.96:0/1375487235 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f05f4036b80 0x7f05f4039090 crc :-1 s=READY pgs=3851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.765 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.764+0000 7f060d59b640 1 --2- 172.21.15.96:0/1375487235 >> v2:172.21.15.96:6801/932229279 conn(0x5645fa455cb0 0x5645fa4e56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:37:53.765 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.766+0000 7f060d59b640 1 --2- 172.21.15.96:0/1375487235 >> v2:172.21.15.96:6801/932229279 conn(0x5645fa455cb0 0x5645fa4e56d0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.774 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.774+0000 7f060d59b640 1 -- 172.21.15.96:0/1375487235 <== osd.0 v2:172.21.15.96:6801/932229279 1 ==== osd_op_reply(1 obj5 [writefull 0~11812] v705'1 uv1 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f05f800a430 con 0x5645fa455cb0 2024-03-21T06:37:53.774 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.774+0000 7f0613c57e00 1 -- 172.21.15.96:0/1375487235 >> v2:172.21.15.96:6801/932229279 conn(0x5645fa455cb0 msgr2=0x5645fa4e56d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.774 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.774+0000 7f0613c57e00 1 --2- 172.21.15.96:0/1375487235 >> v2:172.21.15.96:6801/932229279 conn(0x5645fa455cb0 0x5645fa4e56d0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.774 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.774+0000 7f0613c57e00 1 -- 172.21.15.96:0/1375487235 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f05f4036b80 msgr2=0x7f05f4039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.774 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.774+0000 7f0613c57e00 1 --2- 172.21.15.96:0/1375487235 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f05f4036b80 0x7f05f4039090 crc :-1 s=READY pgs=3851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.774 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.774+0000 7f0613c57e00 1 -- 172.21.15.96:0/1375487235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645fa452640 msgr2=0x5645fa4d4d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.774 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.774+0000 7f0613c57e00 1 --2- 172.21.15.96:0/1375487235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645fa452640 0x5645fa4d4d80 crc :-1 s=READY pgs=12942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.774 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.774+0000 7f0613c57e00 1 -- 172.21.15.96:0/1375487235 shutdown_connections 2024-03-21T06:37:53.774 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.774+0000 7f0613c57e00 1 --2- 172.21.15.96:0/1375487235 >> v2:172.21.15.96:6801/932229279 conn(0x5645fa455cb0 0x5645fa4e56d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.774 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.774+0000 7f0613c57e00 1 --2- 172.21.15.96:0/1375487235 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f05f4036b80 0x7f05f4039090 unknown :-1 s=CLOSED pgs=3851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.774 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.774+0000 7f0613c57e00 1 --2- 172.21.15.96:0/1375487235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645fa452640 0x5645fa4d4d80 unknown :-1 s=CLOSED pgs=12942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.774 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.774+0000 7f0613c57e00 1 -- 172.21.15.96:0/1375487235 >> 172.21.15.96:0/1375487235 conn(0x5645fa361e00 msgr2=0x5645fa44e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.775 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.774+0000 7f0613c57e00 1 -- 172.21.15.96:0/1375487235 shutdown_connections 2024-03-21T06:37:53.775 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.774+0000 7f0613c57e00 1 -- 172.21.15.96:0/1375487235 wait complete. 2024-03-21T06:37:53.776 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:37:53.776 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 416ac5b8-f35e-473b-b584-9498e7bbba57 put obj6 /etc/passwd 2024-03-21T06:37:53.805 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.806+0000 7f81b066ee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.809 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.810+0000 7f81b066ee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.809 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.810+0000 7f81b066ee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.810 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.811+0000 7f81b066ee00 1 Processor -- start 2024-03-21T06:37:53.810 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.811+0000 7f81b066ee00 1 -- start start 2024-03-21T06:37:53.812 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.812+0000 7f81b066ee00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600aea15df0 0x5600aeaa2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.812 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.812+0000 7f81b066ee00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ae91bea0 con 0x5600aea15df0 2024-03-21T06:37:53.812 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.812+0000 7f81a979a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600aea15df0 0x5600aeaa2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.812 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.812+0000 7f81a979a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600aea15df0 0x5600aeaa2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43270/0 (socket says 172.21.15.96:43270) 2024-03-21T06:37:53.812 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.812+0000 7f81a979a640 1 -- 172.21.15.96:0/2945298817 learned_addr learned my addr 172.21.15.96:0/2945298817 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.813 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.812+0000 7f81a979a640 1 -- 172.21.15.96:0/2945298817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600ae8bea30 con 0x5600aea15df0 2024-03-21T06:37:53.813 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.812+0000 7f81a979a640 1 --2- 172.21.15.96:0/2945298817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600aea15df0 0x5600aeaa2530 crc :-1 s=READY pgs=12945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb479336c30986c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.813 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.812+0000 7f819b7fe640 1 -- 172.21.15.96:0/2945298817 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f81a000b500 con 0x5600aea15df0 2024-03-21T06:37:53.813 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.812+0000 7f819b7fe640 1 -- 172.21.15.96:0/2945298817 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f81a000b660 con 0x5600aea15df0 2024-03-21T06:37:53.813 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.812+0000 7f819b7fe640 1 -- 172.21.15.96:0/2945298817 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f81a000b900 con 0x5600aea15df0 2024-03-21T06:37:53.813 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.813+0000 7f81b066ee00 1 -- 172.21.15.96:0/2945298817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600aea15df0 msgr2=0x5600aeaa2530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.813 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.813+0000 7f81b066ee00 1 --2- 172.21.15.96:0/2945298817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600aea15df0 0x5600aeaa2530 crc :-1 s=READY pgs=12945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.813 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.813+0000 7f81b066ee00 1 -- 172.21.15.96:0/2945298817 shutdown_connections 2024-03-21T06:37:53.813 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.813+0000 7f81b066ee00 1 --2- 172.21.15.96:0/2945298817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600aea15df0 0x5600aeaa2530 unknown :-1 s=CLOSED pgs=12945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.813 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.813+0000 7f81b066ee00 1 -- 172.21.15.96:0/2945298817 >> 172.21.15.96:0/2945298817 conn(0x5600ae890e00 msgr2=0x5600ae97d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.813 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.813+0000 7f81b066ee00 1 -- 172.21.15.96:0/2945298817 shutdown_connections 2024-03-21T06:37:53.813 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.813+0000 7f81b066ee00 1 -- 172.21.15.96:0/2945298817 wait complete. 2024-03-21T06:37:53.813 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.813+0000 7f81b066ee00 1 Processor -- start 2024-03-21T06:37:53.813 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.813+0000 7f81b066ee00 1 -- start start 2024-03-21T06:37:53.815 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.814+0000 7f81b066ee00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600aea15df0 0x5600aeaaa750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.816 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.814+0000 7f81b066ee00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ae8bc550 con 0x5600aea15df0 2024-03-21T06:37:53.816 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.814+0000 7f81a979a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600aea15df0 0x5600aeaaa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.816 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.814+0000 7f81a979a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600aea15df0 0x5600aeaaa750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43282/0 (socket says 172.21.15.96:43282) 2024-03-21T06:37:53.816 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.814+0000 7f81a979a640 1 -- 172.21.15.96:0/2150620483 learned_addr learned my addr 172.21.15.96:0/2150620483 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.816 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.814+0000 7f81a979a640 1 -- 172.21.15.96:0/2150620483 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600ae8b1680 con 0x5600aea15df0 2024-03-21T06:37:53.816 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.814+0000 7f81a979a640 1 --2- 172.21.15.96:0/2150620483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600aea15df0 0x5600aeaaa750 crc :-1 s=READY pgs=12946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.816 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.814+0000 7f819a7fc640 1 -- 172.21.15.96:0/2150620483 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f81a00111e0 con 0x5600aea15df0 2024-03-21T06:37:53.816 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.815+0000 7f819a7fc640 1 -- 172.21.15.96:0/2150620483 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f81a0011a80 con 0x5600aea15df0 2024-03-21T06:37:53.816 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.815+0000 7f81b066ee00 1 -- 172.21.15.96:0/2150620483 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5600ae8b10e0 con 0x5600aea15df0 2024-03-21T06:37:53.816 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.815+0000 7f819a7fc640 1 -- 172.21.15.96:0/2150620483 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f81a0011bf0 con 0x5600aea15df0 2024-03-21T06:37:53.816 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.815+0000 7f81b066ee00 1 -- 172.21.15.96:0/2150620483 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600aea9f2f0 con 0x5600aea15df0 2024-03-21T06:37:53.816 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.815+0000 7f819a7fc640 1 -- 172.21.15.96:0/2150620483 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f81a000f450 con 0x5600aea15df0 2024-03-21T06:37:53.816 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.816+0000 7f819a7fc640 1 --2- 172.21.15.96:0/2150620483 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f8188036b30 0x7f8188039040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.816 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.816+0000 7f819a7fc640 1 -- 172.21.15.96:0/2150620483 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(705..705 src has 402..705) v4 ==== 4179+0+0 (crc 0 0 0) 0x7f81a0048580 con 0x5600aea15df0 2024-03-21T06:37:53.817 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.816+0000 7f81b066ee00 1 --2- 172.21.15.96:0/2150620483 >> v2:172.21.15.96:6805/13988474 conn(0x5600aea18d40 0x5600aea14520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.817 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.816+0000 7f81b066ee00 1 -- 172.21.15.96:0/2150620483 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:1 201.4 201:233a42c1:::obj6:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e705) v9 -- 0x5600ae9ff210 con 0x5600aea18d40 2024-03-21T06:37:53.817 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.816+0000 7f81a9f9b640 1 --2- 172.21.15.96:0/2150620483 >> v2:172.21.15.96:6805/13988474 conn(0x5600aea18d40 0x5600aea14520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:37:53.817 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.816+0000 7f81a8f99640 1 --2- 172.21.15.96:0/2150620483 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f8188036b30 0x7f8188039040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.817 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.818+0000 7f81a8f99640 1 --2- 172.21.15.96:0/2150620483 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f8188036b30 0x7f8188039040 crc :-1 s=READY pgs=3852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.818 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.818+0000 7f81a9f9b640 1 --2- 172.21.15.96:0/2150620483 >> v2:172.21.15.96:6805/13988474 conn(0x5600aea18d40 0x5600aea14520 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.826 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.826+0000 7f81a9f9b640 1 -- 172.21.15.96:0/2150620483 <== osd.1 v2:172.21.15.96:6805/13988474 1 ==== osd_op_reply(1 obj6 [writefull 0~11812] v705'1 uv1 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f819c00a430 con 0x5600aea18d40 2024-03-21T06:37:53.826 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.826+0000 7f81b066ee00 1 -- 172.21.15.96:0/2150620483 >> v2:172.21.15.96:6805/13988474 conn(0x5600aea18d40 msgr2=0x5600aea14520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.826 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.826+0000 7f81b066ee00 1 --2- 172.21.15.96:0/2150620483 >> v2:172.21.15.96:6805/13988474 conn(0x5600aea18d40 0x5600aea14520 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.826 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.826+0000 7f81b066ee00 1 -- 172.21.15.96:0/2150620483 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f8188036b30 msgr2=0x7f8188039040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.826 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.826+0000 7f81b066ee00 1 --2- 172.21.15.96:0/2150620483 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f8188036b30 0x7f8188039040 crc :-1 s=READY pgs=3852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.826 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.826+0000 7f81b066ee00 1 -- 172.21.15.96:0/2150620483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600aea15df0 msgr2=0x5600aeaaa750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.826 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.826+0000 7f81b066ee00 1 --2- 172.21.15.96:0/2150620483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600aea15df0 0x5600aeaaa750 crc :-1 s=READY pgs=12946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.827 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.826+0000 7f81b066ee00 1 -- 172.21.15.96:0/2150620483 shutdown_connections 2024-03-21T06:37:53.827 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.826+0000 7f81b066ee00 1 --2- 172.21.15.96:0/2150620483 >> v2:172.21.15.96:6805/13988474 conn(0x5600aea18d40 0x5600aea14520 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.827 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.826+0000 7f81b066ee00 1 --2- 172.21.15.96:0/2150620483 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f8188036b30 0x7f8188039040 unknown :-1 s=CLOSED pgs=3852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.827 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.826+0000 7f81b066ee00 1 --2- 172.21.15.96:0/2150620483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600aea15df0 0x5600aeaaa750 unknown :-1 s=CLOSED pgs=12946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.827 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.826+0000 7f81b066ee00 1 -- 172.21.15.96:0/2150620483 >> 172.21.15.96:0/2150620483 conn(0x5600ae890e00 msgr2=0x5600ae97d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.827 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.826+0000 7f81b066ee00 1 -- 172.21.15.96:0/2150620483 shutdown_connections 2024-03-21T06:37:53.827 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.826+0000 7f81b066ee00 1 -- 172.21.15.96:0/2150620483 wait complete. 2024-03-21T06:37:53.827 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:37:53.827 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 416ac5b8-f35e-473b-b584-9498e7bbba57 put obj7 /etc/passwd 2024-03-21T06:37:53.857 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.858+0000 7f488906ee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.861 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.862+0000 7f488906ee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.861 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.862+0000 7f488906ee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.861 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.862+0000 7f488906ee00 1 Processor -- start 2024-03-21T06:37:53.862 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.862+0000 7f488906ee00 1 -- start start 2024-03-21T06:37:53.864 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.863+0000 7f488906ee00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400cf11640 0x56400cfa5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.864 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.863+0000 7f488906ee00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x56400ceabea0 con 0x56400cf11640 2024-03-21T06:37:53.864 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.863+0000 7f488159a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400cf11640 0x56400cfa5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.864 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.863+0000 7f488159a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400cf11640 0x56400cfa5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43296/0 (socket says 172.21.15.96:43296) 2024-03-21T06:37:53.864 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.863+0000 7f488159a640 1 -- 172.21.15.96:0/4037705957 learned_addr learned my addr 172.21.15.96:0/4037705957 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.864 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.863+0000 7f488159a640 1 -- 172.21.15.96:0/4037705957 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56400ce4ea30 con 0x56400cf11640 2024-03-21T06:37:53.864 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.864+0000 7f488159a640 1 --2- 172.21.15.96:0/4037705957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400cf11640 0x56400cfa5ad0 crc :-1 s=READY pgs=12949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834e9a6a2dbf32df server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.864 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.864+0000 7f487b7fe640 1 -- 172.21.15.96:0/4037705957 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f486c02aeb0 con 0x56400cf11640 2024-03-21T06:37:53.864 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.864+0000 7f487b7fe640 1 -- 172.21.15.96:0/4037705957 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f486c0333d0 con 0x56400cf11640 2024-03-21T06:37:53.864 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.864+0000 7f487b7fe640 1 -- 172.21.15.96:0/4037705957 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f486c033670 con 0x56400cf11640 2024-03-21T06:37:53.864 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.864+0000 7f488906ee00 1 -- 172.21.15.96:0/4037705957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400cf11640 msgr2=0x56400cfa5ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.865 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.864+0000 7f488906ee00 1 --2- 172.21.15.96:0/4037705957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400cf11640 0x56400cfa5ad0 crc :-1 s=READY pgs=12949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.865 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.864+0000 7f488906ee00 1 -- 172.21.15.96:0/4037705957 shutdown_connections 2024-03-21T06:37:53.865 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.864+0000 7f488906ee00 1 --2- 172.21.15.96:0/4037705957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400cf11640 0x56400cfa5ad0 unknown :-1 s=CLOSED pgs=12949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.865 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.864+0000 7f488906ee00 1 -- 172.21.15.96:0/4037705957 >> 172.21.15.96:0/4037705957 conn(0x56400ce20e00 msgr2=0x56400cfab400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.865 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.864+0000 7f488906ee00 1 -- 172.21.15.96:0/4037705957 shutdown_connections 2024-03-21T06:37:53.865 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.864+0000 7f488906ee00 1 -- 172.21.15.96:0/4037705957 wait complete. 2024-03-21T06:37:53.865 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.865+0000 7f488906ee00 1 Processor -- start 2024-03-21T06:37:53.865 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.865+0000 7f488906ee00 1 -- start start 2024-03-21T06:37:53.867 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.866+0000 7f488906ee00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400cf11640 0x56400cfa62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.868 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.866+0000 7f488906ee00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x56400ce4c550 con 0x56400cf11640 2024-03-21T06:37:53.868 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.866+0000 7f488159a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400cf11640 0x56400cfa62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.868 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.866+0000 7f488159a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400cf11640 0x56400cfa62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43302/0 (socket says 172.21.15.96:43302) 2024-03-21T06:37:53.868 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.866+0000 7f488159a640 1 -- 172.21.15.96:0/2172688148 learned_addr learned my addr 172.21.15.96:0/2172688148 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.868 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.866+0000 7f488159a640 1 -- 172.21.15.96:0/2172688148 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56400ce41680 con 0x56400cf11640 2024-03-21T06:37:53.868 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.866+0000 7f488159a640 1 --2- 172.21.15.96:0/2172688148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400cf11640 0x56400cfa62e0 crc :-1 s=READY pgs=12950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.868 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.866+0000 7f487a7fc640 1 -- 172.21.15.96:0/2172688148 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f486c03a040 con 0x56400cf11640 2024-03-21T06:37:53.868 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.866+0000 7f487a7fc640 1 -- 172.21.15.96:0/2172688148 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f486c034440 con 0x56400cf11640 2024-03-21T06:37:53.868 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.866+0000 7f488906ee00 1 -- 172.21.15.96:0/2172688148 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56400ce410e0 con 0x56400cf11640 2024-03-21T06:37:53.868 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.866+0000 7f487a7fc640 1 -- 172.21.15.96:0/2172688148 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f486c0346e0 con 0x56400cf11640 2024-03-21T06:37:53.868 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.866+0000 7f488906ee00 1 -- 172.21.15.96:0/2172688148 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56400cefb440 con 0x56400cf11640 2024-03-21T06:37:53.868 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.867+0000 7f487a7fc640 1 -- 172.21.15.96:0/2172688148 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f486c031040 con 0x56400cf11640 2024-03-21T06:37:53.868 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.867+0000 7f487a7fc640 1 --2- 172.21.15.96:0/2172688148 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4868036b80 0x7f4868039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.868 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.867+0000 7f487a7fc640 1 -- 172.21.15.96:0/2172688148 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(705..705 src has 402..705) v4 ==== 4179+0+0 (crc 0 0 0) 0x7f486c02e040 con 0x56400cf11640 2024-03-21T06:37:53.869 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.867+0000 7f488906ee00 1 --2- 172.21.15.96:0/2172688148 >> v2:172.21.15.96:6805/13988474 conn(0x56400cf14ae0 0x56400cefa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.869 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.867+0000 7f488906ee00 1 -- 172.21.15.96:0/2172688148 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:1 201.9 201:9a09113a:::obj7:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e705) v9 -- 0x56400d035360 con 0x56400cf14ae0 2024-03-21T06:37:53.869 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.868+0000 7f4880d99640 1 --2- 172.21.15.96:0/2172688148 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4868036b80 0x7f4868039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.869 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.868+0000 7f4881d9b640 1 --2- 172.21.15.96:0/2172688148 >> v2:172.21.15.96:6805/13988474 conn(0x56400cf14ae0 0x56400cefa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:37:53.869 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.868+0000 7f4880d99640 1 --2- 172.21.15.96:0/2172688148 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4868036b80 0x7f4868039090 crc :-1 s=READY pgs=3853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.869 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.870+0000 7f4881d9b640 1 --2- 172.21.15.96:0/2172688148 >> v2:172.21.15.96:6805/13988474 conn(0x56400cf14ae0 0x56400cefa950 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.877 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.877+0000 7f4881d9b640 1 -- 172.21.15.96:0/2172688148 <== osd.1 v2:172.21.15.96:6805/13988474 1 ==== osd_op_reply(1 obj7 [writefull 0~11812] v705'1 uv1 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f487400c430 con 0x56400cf14ae0 2024-03-21T06:37:53.877 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.877+0000 7f488906ee00 1 -- 172.21.15.96:0/2172688148 >> v2:172.21.15.96:6805/13988474 conn(0x56400cf14ae0 msgr2=0x56400cefa950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.877 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.877+0000 7f488906ee00 1 --2- 172.21.15.96:0/2172688148 >> v2:172.21.15.96:6805/13988474 conn(0x56400cf14ae0 0x56400cefa950 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.877 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.877+0000 7f488906ee00 1 -- 172.21.15.96:0/2172688148 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4868036b80 msgr2=0x7f4868039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.877 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.877+0000 7f488906ee00 1 --2- 172.21.15.96:0/2172688148 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4868036b80 0x7f4868039090 crc :-1 s=READY pgs=3853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.877 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.877+0000 7f488906ee00 1 -- 172.21.15.96:0/2172688148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400cf11640 msgr2=0x56400cfa62e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.877 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.877+0000 7f488906ee00 1 --2- 172.21.15.96:0/2172688148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400cf11640 0x56400cfa62e0 crc :-1 s=READY pgs=12950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.877 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.877+0000 7f488906ee00 1 -- 172.21.15.96:0/2172688148 shutdown_connections 2024-03-21T06:37:53.877 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.877+0000 7f488906ee00 1 --2- 172.21.15.96:0/2172688148 >> v2:172.21.15.96:6805/13988474 conn(0x56400cf14ae0 0x56400cefa950 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.877 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.877+0000 7f488906ee00 1 --2- 172.21.15.96:0/2172688148 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4868036b80 0x7f4868039090 unknown :-1 s=CLOSED pgs=3853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.877 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.877+0000 7f488906ee00 1 --2- 172.21.15.96:0/2172688148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56400cf11640 0x56400cfa62e0 unknown :-1 s=CLOSED pgs=12950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.877 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.877+0000 7f488906ee00 1 -- 172.21.15.96:0/2172688148 >> 172.21.15.96:0/2172688148 conn(0x56400ce20e00 msgr2=0x56400cfab400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.878 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.877+0000 7f488906ee00 1 -- 172.21.15.96:0/2172688148 shutdown_connections 2024-03-21T06:37:53.878 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.877+0000 7f488906ee00 1 -- 172.21.15.96:0/2172688148 wait complete. 2024-03-21T06:37:53.878 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:37:53.878 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 416ac5b8-f35e-473b-b584-9498e7bbba57 put obj8 /etc/passwd 2024-03-21T06:37:53.908 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.909+0000 7f92d4276e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.912 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.913+0000 7f92d4276e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.913 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.914+0000 7f92d4276e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.913 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.914+0000 7f92d4276e00 1 Processor -- start 2024-03-21T06:37:53.913 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.914+0000 7f92d4276e00 1 -- start start 2024-03-21T06:37:53.915 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.915+0000 7f92d4276e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7817f9df0 0x55e781886530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.915 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.915+0000 7f92d4276e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7816ffea0 con 0x55e7817f9df0 2024-03-21T06:37:53.915 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.915+0000 7f92c7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7817f9df0 0x55e781886530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.915 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.915+0000 7f92c7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7817f9df0 0x55e781886530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43318/0 (socket says 172.21.15.96:43318) 2024-03-21T06:37:53.915 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.915+0000 7f92c7fff640 1 -- 172.21.15.96:0/1279366536 learned_addr learned my addr 172.21.15.96:0/1279366536 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.915 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.915+0000 7f92c7fff640 1 -- 172.21.15.96:0/1279366536 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7816a2a30 con 0x55e7817f9df0 2024-03-21T06:37:53.915 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.915+0000 7f92c7fff640 1 --2- 172.21.15.96:0/1279366536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7817f9df0 0x55e781886530 crc :-1 s=READY pgs=12953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44829872a1c5c11 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.915 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.915+0000 7f92c67fc640 1 -- 172.21.15.96:0/1279366536 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f92b80061e0 con 0x55e7817f9df0 2024-03-21T06:37:53.916 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.915+0000 7f92c67fc640 1 -- 172.21.15.96:0/1279366536 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f92b8006340 con 0x55e7817f9df0 2024-03-21T06:37:53.916 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.915+0000 7f92c67fc640 1 -- 172.21.15.96:0/1279366536 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f92b8001b40 con 0x55e7817f9df0 2024-03-21T06:37:53.916 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.916+0000 7f92d4276e00 1 -- 172.21.15.96:0/1279366536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7817f9df0 msgr2=0x55e781886530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.916 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.916+0000 7f92d4276e00 1 --2- 172.21.15.96:0/1279366536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7817f9df0 0x55e781886530 crc :-1 s=READY pgs=12953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.916 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.916+0000 7f92d4276e00 1 -- 172.21.15.96:0/1279366536 shutdown_connections 2024-03-21T06:37:53.916 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.916+0000 7f92d4276e00 1 --2- 172.21.15.96:0/1279366536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7817f9df0 0x55e781886530 unknown :-1 s=CLOSED pgs=12953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.916 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.916+0000 7f92d4276e00 1 -- 172.21.15.96:0/1279366536 >> 172.21.15.96:0/1279366536 conn(0x55e781674e00 msgr2=0x55e781761300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.916 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.916+0000 7f92d4276e00 1 -- 172.21.15.96:0/1279366536 shutdown_connections 2024-03-21T06:37:53.916 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.916+0000 7f92d4276e00 1 -- 172.21.15.96:0/1279366536 wait complete. 2024-03-21T06:37:53.918 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.916+0000 7f92d4276e00 1 Processor -- start 2024-03-21T06:37:53.919 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.917+0000 7f92d4276e00 1 -- start start 2024-03-21T06:37:53.919 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.917+0000 7f92d4276e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7817f9df0 0x55e7817f9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.919 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.917+0000 7f92d4276e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7816a0550 con 0x55e7817f9df0 2024-03-21T06:37:53.919 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.917+0000 7f92c7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7817f9df0 0x55e7817f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.919 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.917+0000 7f92c7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7817f9df0 0x55e7817f9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43330/0 (socket says 172.21.15.96:43330) 2024-03-21T06:37:53.919 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.917+0000 7f92c7fff640 1 -- 172.21.15.96:0/179895477 learned_addr learned my addr 172.21.15.96:0/179895477 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.919 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.917+0000 7f92c7fff640 1 -- 172.21.15.96:0/179895477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e781695680 con 0x55e7817f9df0 2024-03-21T06:37:53.919 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.918+0000 7f92c7fff640 1 --2- 172.21.15.96:0/179895477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7817f9df0 0x55e7817f9250 crc :-1 s=READY pgs=12954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.919 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.918+0000 7f92c57fa640 1 -- 172.21.15.96:0/179895477 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f92b8004a20 con 0x55e7817f9df0 2024-03-21T06:37:53.919 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.918+0000 7f92c57fa640 1 -- 172.21.15.96:0/179895477 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f92b8004b80 con 0x55e7817f9df0 2024-03-21T06:37:53.920 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.918+0000 7f92d4276e00 1 -- 172.21.15.96:0/179895477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e7816950e0 con 0x55e7817f9df0 2024-03-21T06:37:53.920 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.918+0000 7f92c57fa640 1 -- 172.21.15.96:0/179895477 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f92b8004e20 con 0x55e7817f9df0 2024-03-21T06:37:53.920 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.918+0000 7f92d4276e00 1 -- 172.21.15.96:0/179895477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7817650c0 con 0x55e7817f9df0 2024-03-21T06:37:53.920 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.919+0000 7f92c57fa640 1 -- 172.21.15.96:0/179895477 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f92b8014410 con 0x55e7817f9df0 2024-03-21T06:37:53.920 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.919+0000 7f92c57fa640 1 --2- 172.21.15.96:0/179895477 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f92b4036b30 0x7f92b4039040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.920 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.919+0000 7f92c57fa640 1 -- 172.21.15.96:0/179895477 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(705..705 src has 402..705) v4 ==== 4179+0+0 (crc 0 0 0) 0x7f92b80480d0 con 0x55e7817f9df0 2024-03-21T06:37:53.920 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.919+0000 7f92d4276e00 1 --2- 172.21.15.96:0/179895477 >> v2:172.21.15.96:6805/13988474 conn(0x55e7817fcd40 0x55e781886cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.920 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.919+0000 7f92d4276e00 1 -- 172.21.15.96:0/179895477 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:1 201.5 201:a557efb1:::obj8:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e705) v9 -- 0x55e781764830 con 0x55e7817fcd40 2024-03-21T06:37:53.920 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.919+0000 7f92c77fe640 1 --2- 172.21.15.96:0/179895477 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f92b4036b30 0x7f92b4039040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.920 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.920+0000 7f92ccf9b640 1 --2- 172.21.15.96:0/179895477 >> v2:172.21.15.96:6805/13988474 conn(0x55e7817fcd40 0x55e781886cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:37:53.920 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.920+0000 7f92c77fe640 1 --2- 172.21.15.96:0/179895477 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f92b4036b30 0x7f92b4039040 crc :-1 s=READY pgs=3854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.921 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.921+0000 7f92ccf9b640 1 --2- 172.21.15.96:0/179895477 >> v2:172.21.15.96:6805/13988474 conn(0x55e7817fcd40 0x55e781886cb0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.929 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.929+0000 7f92ccf9b640 1 -- 172.21.15.96:0/179895477 <== osd.1 v2:172.21.15.96:6805/13988474 1 ==== osd_op_reply(1 obj8 [writefull 0~11812] v705'1 uv1 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f92c000c430 con 0x55e7817fcd40 2024-03-21T06:37:53.929 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.929+0000 7f92d4276e00 1 -- 172.21.15.96:0/179895477 >> v2:172.21.15.96:6805/13988474 conn(0x55e7817fcd40 msgr2=0x55e781886cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.929 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.929+0000 7f92d4276e00 1 --2- 172.21.15.96:0/179895477 >> v2:172.21.15.96:6805/13988474 conn(0x55e7817fcd40 0x55e781886cb0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.929 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.929+0000 7f92d4276e00 1 -- 172.21.15.96:0/179895477 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f92b4036b30 msgr2=0x7f92b4039040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.930 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.929+0000 7f92d4276e00 1 --2- 172.21.15.96:0/179895477 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f92b4036b30 0x7f92b4039040 crc :-1 s=READY pgs=3854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.930 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.929+0000 7f92d4276e00 1 -- 172.21.15.96:0/179895477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7817f9df0 msgr2=0x55e7817f9250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.930 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.929+0000 7f92d4276e00 1 --2- 172.21.15.96:0/179895477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7817f9df0 0x55e7817f9250 crc :-1 s=READY pgs=12954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.930 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.929+0000 7f92d4276e00 1 -- 172.21.15.96:0/179895477 shutdown_connections 2024-03-21T06:37:53.930 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.929+0000 7f92d4276e00 1 --2- 172.21.15.96:0/179895477 >> v2:172.21.15.96:6805/13988474 conn(0x55e7817fcd40 0x55e781886cb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.930 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.929+0000 7f92d4276e00 1 --2- 172.21.15.96:0/179895477 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f92b4036b30 0x7f92b4039040 unknown :-1 s=CLOSED pgs=3854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.930 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.929+0000 7f92d4276e00 1 --2- 172.21.15.96:0/179895477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7817f9df0 0x55e7817f9250 unknown :-1 s=CLOSED pgs=12954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.930 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.930+0000 7f92d4276e00 1 -- 172.21.15.96:0/179895477 >> 172.21.15.96:0/179895477 conn(0x55e781674e00 msgr2=0x55e781761300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.930 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.930+0000 7f92d4276e00 1 -- 172.21.15.96:0/179895477 shutdown_connections 2024-03-21T06:37:53.930 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.930+0000 7f92d4276e00 1 -- 172.21.15.96:0/179895477 wait complete. 2024-03-21T06:37:53.931 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:37:53.931 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 416ac5b8-f35e-473b-b584-9498e7bbba57 put obj9 /etc/passwd 2024-03-21T06:37:53.961 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.961+0000 7f9f05677e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.964 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.966+0000 7f9f05677e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.965 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.966+0000 7f9f05677e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:53.965 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.966+0000 7f9f05677e00 1 Processor -- start 2024-03-21T06:37:53.965 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.966+0000 7f9f05677e00 1 -- start start 2024-03-21T06:37:53.966 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.967+0000 7f9f05677e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b5b843df0 0x556b5b8d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.966 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.967+0000 7f9f05677e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b5b749ea0 con 0x556b5b843df0 2024-03-21T06:37:53.966 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.967+0000 7f9efdb9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b5b843df0 0x556b5b8d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.967 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.967+0000 7f9efdb9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b5b843df0 0x556b5b8d0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43352/0 (socket says 172.21.15.96:43352) 2024-03-21T06:37:53.967 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.967+0000 7f9efdb9a640 1 -- 172.21.15.96:0/579248573 learned_addr learned my addr 172.21.15.96:0/579248573 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.967 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.967+0000 7f9efdb9a640 1 -- 172.21.15.96:0/579248573 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b5b6eca30 con 0x556b5b843df0 2024-03-21T06:37:53.967 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.967+0000 7f9efdb9a640 1 --2- 172.21.15.96:0/579248573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b5b843df0 0x556b5b8d0530 crc :-1 s=READY pgs=12957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef1fb5fc59631bd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.967 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.967+0000 7f9eeffff640 1 -- 172.21.15.96:0/579248573 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9ef400b500 con 0x556b5b843df0 2024-03-21T06:37:53.967 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.967+0000 7f9eeffff640 1 -- 172.21.15.96:0/579248573 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f9ef400b660 con 0x556b5b843df0 2024-03-21T06:37:53.967 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.967+0000 7f9eeffff640 1 -- 172.21.15.96:0/579248573 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9ef400b900 con 0x556b5b843df0 2024-03-21T06:37:53.969 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.968+0000 7f9f05677e00 1 -- 172.21.15.96:0/579248573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b5b843df0 msgr2=0x556b5b8d0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.969 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.968+0000 7f9f05677e00 1 --2- 172.21.15.96:0/579248573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b5b843df0 0x556b5b8d0530 crc :-1 s=READY pgs=12957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.969 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.968+0000 7f9f05677e00 1 -- 172.21.15.96:0/579248573 shutdown_connections 2024-03-21T06:37:53.969 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.968+0000 7f9f05677e00 1 --2- 172.21.15.96:0/579248573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b5b843df0 0x556b5b8d0530 unknown :-1 s=CLOSED pgs=12957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.969 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.968+0000 7f9f05677e00 1 -- 172.21.15.96:0/579248573 >> 172.21.15.96:0/579248573 conn(0x556b5b6bee00 msgr2=0x556b5b7ab300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.969 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.968+0000 7f9f05677e00 1 -- 172.21.15.96:0/579248573 shutdown_connections 2024-03-21T06:37:53.969 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.968+0000 7f9f05677e00 1 -- 172.21.15.96:0/579248573 wait complete. 2024-03-21T06:37:53.969 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.969+0000 7f9f05677e00 1 Processor -- start 2024-03-21T06:37:53.969 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.969+0000 7f9f05677e00 1 -- start start 2024-03-21T06:37:53.970 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.969+0000 7f9f05677e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b5b843df0 0x556b5b843250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.970 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.969+0000 7f9f05677e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b5b6ea550 con 0x556b5b843df0 2024-03-21T06:37:53.970 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.969+0000 7f9efdb9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b5b843df0 0x556b5b843250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.970 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.969+0000 7f9efdb9a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b5b843df0 0x556b5b843250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43366/0 (socket says 172.21.15.96:43366) 2024-03-21T06:37:53.970 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.969+0000 7f9efdb9a640 1 -- 172.21.15.96:0/2408333303 learned_addr learned my addr 172.21.15.96:0/2408333303 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:53.970 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.970+0000 7f9efdb9a640 1 -- 172.21.15.96:0/2408333303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b5b6df680 con 0x556b5b843df0 2024-03-21T06:37:53.970 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.970+0000 7f9efdb9a640 1 --2- 172.21.15.96:0/2408333303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b5b843df0 0x556b5b843250 crc :-1 s=READY pgs=12958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.970 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.970+0000 7f9eeeffd640 1 -- 172.21.15.96:0/2408333303 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9ef40111e0 con 0x556b5b843df0 2024-03-21T06:37:53.970 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.970+0000 7f9eeeffd640 1 -- 172.21.15.96:0/2408333303 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f9ef4011a80 con 0x556b5b843df0 2024-03-21T06:37:53.970 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.970+0000 7f9eeeffd640 1 -- 172.21.15.96:0/2408333303 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f9ef4011bf0 con 0x556b5b843df0 2024-03-21T06:37:53.970 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.970+0000 7f9f05677e00 1 -- 172.21.15.96:0/2408333303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556b5b6df0e0 con 0x556b5b843df0 2024-03-21T06:37:53.970 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.970+0000 7f9f05677e00 1 -- 172.21.15.96:0/2408333303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b5b8ced70 con 0x556b5b843df0 2024-03-21T06:37:53.971 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.971+0000 7f9eeeffd640 1 -- 172.21.15.96:0/2408333303 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f9ef400f450 con 0x556b5b843df0 2024-03-21T06:37:53.971 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.971+0000 7f9eeeffd640 1 --2- 172.21.15.96:0/2408333303 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f9edc036b80 0x7f9edc039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.972 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.971+0000 7f9eeeffd640 1 -- 172.21.15.96:0/2408333303 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(705..705 src has 402..705) v4 ==== 4179+0+0 (crc 0 0 0) 0x7f9ef40485e0 con 0x556b5b843df0 2024-03-21T06:37:53.972 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.971+0000 7f9f05677e00 1 --2- 172.21.15.96:0/2408333303 >> v2:172.21.15.184:6811/276480837 conn(0x556b5b846d40 0x556b5b7ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:53.972 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.971+0000 7f9f05677e00 1 -- 172.21.15.96:0/2408333303 --> v2:172.21.15.184:6811/276480837 -- osd_op(unknown.0.0:1 201.7 201:fe351e27:::obj9:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e705) v9 -- 0x556b5b8ce0c0 con 0x556b5b846d40 2024-03-21T06:37:53.972 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.971+0000 7f9efd399640 1 --2- 172.21.15.96:0/2408333303 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f9edc036b80 0x7f9edc039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:53.972 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.972+0000 7f9efd399640 1 --2- 172.21.15.96:0/2408333303 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f9edc036b80 0x7f9edc039090 crc :-1 s=READY pgs=3855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.972 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.972+0000 7f9efe39b640 1 --2- 172.21.15.96:0/2408333303 >> v2:172.21.15.184:6811/276480837 conn(0x556b5b846d40 0x556b5b7ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:37:53.974 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.975+0000 7f9efe39b640 1 --2- 172.21.15.96:0/2408333303 >> v2:172.21.15.184:6811/276480837 conn(0x556b5b846d40 0x556b5b7ae0c0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:53.983 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.984+0000 7f9efe39b640 1 -- 172.21.15.96:0/2408333303 <== osd.3 v2:172.21.15.184:6811/276480837 1 ==== osd_op_reply(1 obj9 [writefull 0~11812] v705'2 uv2 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f9ef000a430 con 0x556b5b846d40 2024-03-21T06:37:53.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.984+0000 7f9f05677e00 1 -- 172.21.15.96:0/2408333303 >> v2:172.21.15.184:6811/276480837 conn(0x556b5b846d40 msgr2=0x556b5b7ae0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.984+0000 7f9f05677e00 1 --2- 172.21.15.96:0/2408333303 >> v2:172.21.15.184:6811/276480837 conn(0x556b5b846d40 0x556b5b7ae0c0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.984+0000 7f9f05677e00 1 -- 172.21.15.96:0/2408333303 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f9edc036b80 msgr2=0x7f9edc039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.984+0000 7f9f05677e00 1 --2- 172.21.15.96:0/2408333303 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f9edc036b80 0x7f9edc039090 crc :-1 s=READY pgs=3855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.984+0000 7f9f05677e00 1 -- 172.21.15.96:0/2408333303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b5b843df0 msgr2=0x556b5b843250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:53.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.984+0000 7f9f05677e00 1 --2- 172.21.15.96:0/2408333303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b5b843df0 0x556b5b843250 crc :-1 s=READY pgs=12958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.984+0000 7f9f05677e00 1 -- 172.21.15.96:0/2408333303 shutdown_connections 2024-03-21T06:37:53.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.984+0000 7f9f05677e00 1 --2- 172.21.15.96:0/2408333303 >> v2:172.21.15.184:6811/276480837 conn(0x556b5b846d40 0x556b5b7ae0c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.984+0000 7f9f05677e00 1 --2- 172.21.15.96:0/2408333303 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f9edc036b80 0x7f9edc039090 unknown :-1 s=CLOSED pgs=3855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.984+0000 7f9f05677e00 1 --2- 172.21.15.96:0/2408333303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556b5b843df0 0x556b5b843250 unknown :-1 s=CLOSED pgs=12958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:53.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.984+0000 7f9f05677e00 1 -- 172.21.15.96:0/2408333303 >> 172.21.15.96:0/2408333303 conn(0x556b5b6bee00 msgr2=0x556b5b7ab300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:53.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.984+0000 7f9f05677e00 1 -- 172.21.15.96:0/2408333303 shutdown_connections 2024-03-21T06:37:53.984 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:53.984+0000 7f9f05677e00 1 -- 172.21.15.96:0/2408333303 wait complete. 2024-03-21T06:37:53.985 INFO:tasks.workunit.client.0.smithi096.stderr:+ for f in `seq 1 10` 2024-03-21T06:37:53.985 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 416ac5b8-f35e-473b-b584-9498e7bbba57 put obj10 /etc/passwd 2024-03-21T06:37:54.014 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.015+0000 7efe74086e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:54.018 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.019+0000 7efe74086e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:54.018 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.019+0000 7efe74086e00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:37:54.019 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.020+0000 7efe74086e00 1 Processor -- start 2024-03-21T06:37:54.019 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.020+0000 7efe74086e00 1 -- start start 2024-03-21T06:37:54.021 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.021+0000 7efe74086e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5585ffd61df0 0x5585ffdef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:54.021 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.021+0000 7efe74086e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ffc67ea0 con 0x5585ffd61df0 2024-03-21T06:37:54.021 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.021+0000 7efe6d19a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5585ffd61df0 0x5585ffdef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:54.022 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.021+0000 7efe6d19a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5585ffd61df0 0x5585ffdef3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43382/0 (socket says 172.21.15.96:43382) 2024-03-21T06:37:54.022 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.021+0000 7efe6d19a640 1 -- 172.21.15.96:0/27448030 learned_addr learned my addr 172.21.15.96:0/27448030 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:54.022 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.021+0000 7efe6d19a640 1 -- 172.21.15.96:0/27448030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585ffc0aa30 con 0x5585ffd61df0 2024-03-21T06:37:54.022 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.021+0000 7efe6d19a640 1 --2- 172.21.15.96:0/27448030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5585ffd61df0 0x5585ffdef3f0 crc :-1 s=READY pgs=12961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3512a858638aacd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:54.022 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.021+0000 7efe5f7fe640 1 -- 172.21.15.96:0/27448030 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efe6400b5d0 con 0x5585ffd61df0 2024-03-21T06:37:54.022 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.021+0000 7efe5f7fe640 1 -- 172.21.15.96:0/27448030 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7efe6400b730 con 0x5585ffd61df0 2024-03-21T06:37:54.022 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.021+0000 7efe5f7fe640 1 -- 172.21.15.96:0/27448030 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efe6400b9d0 con 0x5585ffd61df0 2024-03-21T06:37:54.022 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.022+0000 7efe74086e00 1 -- 172.21.15.96:0/27448030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5585ffd61df0 msgr2=0x5585ffdef3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:54.022 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.022+0000 7efe74086e00 1 --2- 172.21.15.96:0/27448030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5585ffd61df0 0x5585ffdef3f0 crc :-1 s=READY pgs=12961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:54.022 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.022+0000 7efe74086e00 1 -- 172.21.15.96:0/27448030 shutdown_connections 2024-03-21T06:37:54.022 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.022+0000 7efe74086e00 1 --2- 172.21.15.96:0/27448030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5585ffd61df0 0x5585ffdef3f0 unknown :-1 s=CLOSED pgs=12961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:54.022 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.022+0000 7efe74086e00 1 -- 172.21.15.96:0/27448030 >> 172.21.15.96:0/27448030 conn(0x5585ffbdce00 msgr2=0x5585ffcc92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:54.022 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.022+0000 7efe74086e00 1 -- 172.21.15.96:0/27448030 shutdown_connections 2024-03-21T06:37:54.022 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.022+0000 7efe74086e00 1 -- 172.21.15.96:0/27448030 wait complete. 2024-03-21T06:37:54.023 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.022+0000 7efe74086e00 1 Processor -- start 2024-03-21T06:37:54.023 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.022+0000 7efe74086e00 1 -- start start 2024-03-21T06:37:54.025 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.023+0000 7efe74086e00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5585ffd61df0 0x5585ffdf6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:54.025 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.023+0000 7efe74086e00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ffc08550 con 0x5585ffd61df0 2024-03-21T06:37:54.025 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.023+0000 7efe6d19a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5585ffd61df0 0x5585ffdf6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:54.025 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.023+0000 7efe6d19a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5585ffd61df0 0x5585ffdf6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43398/0 (socket says 172.21.15.96:43398) 2024-03-21T06:37:54.025 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.023+0000 7efe6d19a640 1 -- 172.21.15.96:0/3712783946 learned_addr learned my addr 172.21.15.96:0/3712783946 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:37:54.026 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.023+0000 7efe6d19a640 1 -- 172.21.15.96:0/3712783946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585ffbfd680 con 0x5585ffd61df0 2024-03-21T06:37:54.026 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.023+0000 7efe6d19a640 1 --2- 172.21.15.96:0/3712783946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5585ffd61df0 0x5585ffdf6d80 crc :-1 s=READY pgs=12962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:54.026 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.023+0000 7efe5e7fc640 1 -- 172.21.15.96:0/3712783946 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efe64012910 con 0x5585ffd61df0 2024-03-21T06:37:54.026 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.024+0000 7efe5e7fc640 1 -- 172.21.15.96:0/3712783946 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7efe64005040 con 0x5585ffd61df0 2024-03-21T06:37:54.026 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.024+0000 7efe74086e00 1 -- 172.21.15.96:0/3712783946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5585ffbfd0e0 con 0x5585ffd61df0 2024-03-21T06:37:54.026 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.024+0000 7efe5e7fc640 1 -- 172.21.15.96:0/3712783946 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efe64012ba0 con 0x5585ffd61df0 2024-03-21T06:37:54.026 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.024+0000 7efe74086e00 1 -- 172.21.15.96:0/3712783946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585ffdec6a0 con 0x5585ffd61df0 2024-03-21T06:37:54.026 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.024+0000 7efe5e7fc640 1 -- 172.21.15.96:0/3712783946 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7efe6400d040 con 0x5585ffd61df0 2024-03-21T06:37:54.026 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.025+0000 7efe5e7fc640 1 --2- 172.21.15.96:0/3712783946 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7efe4c036b80 0x7efe4c039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:54.026 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.025+0000 7efe5e7fc640 1 -- 172.21.15.96:0/3712783946 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(705..705 src has 402..705) v4 ==== 4179+0+0 (crc 0 0 0) 0x7efe6400a040 con 0x5585ffd61df0 2024-03-21T06:37:54.026 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.025+0000 7efe74086e00 1 --2- 172.21.15.96:0/3712783946 >> v2:172.21.15.96:6805/13988474 conn(0x5585ffd64d40 0x5585ffdee700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:37:54.026 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.025+0000 7efe74086e00 1 -- 172.21.15.96:0/3712783946 --> v2:172.21.15.96:6805/13988474 -- osd_op(unknown.0.0:1 201.b 201:d1337354:::obj10:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e705) v9 -- 0x5585ffd4b1f0 con 0x5585ffd64d40 2024-03-21T06:37:54.026 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.025+0000 7efe6c999640 1 --2- 172.21.15.96:0/3712783946 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7efe4c036b80 0x7efe4c039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:37:54.027 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.025+0000 7efe6d99b640 1 --2- 172.21.15.96:0/3712783946 >> v2:172.21.15.96:6805/13988474 conn(0x5585ffd64d40 0x5585ffdee700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:37:54.027 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.026+0000 7efe6c999640 1 --2- 172.21.15.96:0/3712783946 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7efe4c036b80 0x7efe4c039090 crc :-1 s=READY pgs=3856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:54.027 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.027+0000 7efe6d99b640 1 --2- 172.21.15.96:0/3712783946 >> v2:172.21.15.96:6805/13988474 conn(0x5585ffd64d40 0x5585ffdee700 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:37:54.035 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.035+0000 7efe6d99b640 1 -- 172.21.15.96:0/3712783946 <== osd.1 v2:172.21.15.96:6805/13988474 1 ==== osd_op_reply(1 obj10 [writefull 0~11812] v705'1 uv1 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7efe6000a430 con 0x5585ffd64d40 2024-03-21T06:37:54.035 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.035+0000 7efe74086e00 1 -- 172.21.15.96:0/3712783946 >> v2:172.21.15.96:6805/13988474 conn(0x5585ffd64d40 msgr2=0x5585ffdee700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:54.035 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.035+0000 7efe74086e00 1 --2- 172.21.15.96:0/3712783946 >> v2:172.21.15.96:6805/13988474 conn(0x5585ffd64d40 0x5585ffdee700 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:54.035 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.035+0000 7efe74086e00 1 -- 172.21.15.96:0/3712783946 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7efe4c036b80 msgr2=0x7efe4c039090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:54.035 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.035+0000 7efe74086e00 1 --2- 172.21.15.96:0/3712783946 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7efe4c036b80 0x7efe4c039090 crc :-1 s=READY pgs=3856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:54.035 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.035+0000 7efe74086e00 1 -- 172.21.15.96:0/3712783946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5585ffd61df0 msgr2=0x5585ffdf6d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:37:54.035 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.035+0000 7efe74086e00 1 --2- 172.21.15.96:0/3712783946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5585ffd61df0 0x5585ffdf6d80 crc :-1 s=READY pgs=12962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:54.035 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.035+0000 7efe74086e00 1 -- 172.21.15.96:0/3712783946 shutdown_connections 2024-03-21T06:37:54.035 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.035+0000 7efe74086e00 1 --2- 172.21.15.96:0/3712783946 >> v2:172.21.15.96:6805/13988474 conn(0x5585ffd64d40 0x5585ffdee700 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:54.035 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.035+0000 7efe74086e00 1 --2- 172.21.15.96:0/3712783946 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7efe4c036b80 0x7efe4c039090 unknown :-1 s=CLOSED pgs=3856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:54.035 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.035+0000 7efe74086e00 1 --2- 172.21.15.96:0/3712783946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5585ffd61df0 0x5585ffdf6d80 unknown :-1 s=CLOSED pgs=12962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:37:54.035 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.035+0000 7efe74086e00 1 -- 172.21.15.96:0/3712783946 >> 172.21.15.96:0/3712783946 conn(0x5585ffbdce00 msgr2=0x5585ffcc92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:37:54.036 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.035+0000 7efe74086e00 1 -- 172.21.15.96:0/3712783946 shutdown_connections 2024-03-21T06:37:54.036 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:37:54.035+0000 7efe74086e00 1 -- 172.21.15.96:0/3712783946 wait complete. 2024-03-21T06:37:54.036 INFO:tasks.workunit.client.0.smithi096.stderr:+ sleep 30 2024-03-21T06:38:24.039 INFO:tasks.workunit.client.0.smithi096.stderr:+ rados -p 13dbd6ab-5566-4934-957c-dcb5f3ce8145 put threemore /etc/passwd 2024-03-21T06:38:24.075 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.075+0000 7fee448bee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:38:24.079 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.079+0000 7fee448bee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:38:24.079 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.079+0000 7fee448bee00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:38:24.080 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.080+0000 7fee448bee00 1 Processor -- start 2024-03-21T06:38:24.080 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.080+0000 7fee448bee00 1 -- start start 2024-03-21T06:38:24.082 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.080+0000 7fee448bee00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f1a4c5640 0x556f1a5e6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:38:24.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.080+0000 7fee448bee00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1a45fea0 con 0x556f1a4c5640 2024-03-21T06:38:24.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.080+0000 7fee3d99a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f1a4c5640 0x556f1a5e6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:38:24.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.081+0000 7fee3d99a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f1a4c5640 0x556f1a5e6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55810/0 (socket says 172.21.15.96:55810) 2024-03-21T06:38:24.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.081+0000 7fee3d99a640 1 -- 172.21.15.96:0/1917219493 learned_addr learned my addr 172.21.15.96:0/1917219493 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:38:24.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.081+0000 7fee3d99a640 1 -- 172.21.15.96:0/1917219493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f1a402a30 con 0x556f1a4c5640 2024-03-21T06:38:24.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.081+0000 7fee3d99a640 1 --2- 172.21.15.96:0/1917219493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f1a4c5640 0x556f1a5e6ab0 crc :-1 s=READY pgs=12965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259657044bfcdd02 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:38:24.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.081+0000 7fee2ffff640 1 -- 172.21.15.96:0/1917219493 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fee34029960 con 0x556f1a4c5640 2024-03-21T06:38:24.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.081+0000 7fee2ffff640 1 -- 172.21.15.96:0/1917219493 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fee340333d0 con 0x556f1a4c5640 2024-03-21T06:38:24.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.081+0000 7fee2ffff640 1 -- 172.21.15.96:0/1917219493 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fee34033670 con 0x556f1a4c5640 2024-03-21T06:38:24.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.082+0000 7fee448bee00 1 -- 172.21.15.96:0/1917219493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f1a4c5640 msgr2=0x556f1a5e6ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:24.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.082+0000 7fee448bee00 1 --2- 172.21.15.96:0/1917219493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f1a4c5640 0x556f1a5e6ab0 crc :-1 s=READY pgs=12965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:24.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.082+0000 7fee448bee00 1 -- 172.21.15.96:0/1917219493 shutdown_connections 2024-03-21T06:38:24.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.082+0000 7fee448bee00 1 --2- 172.21.15.96:0/1917219493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f1a4c5640 0x556f1a5e6ab0 unknown :-1 s=CLOSED pgs=12965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:24.083 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.082+0000 7fee448bee00 1 -- 172.21.15.96:0/1917219493 >> 172.21.15.96:0/1917219493 conn(0x556f1a3d4e00 msgr2=0x556f1a4c3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:38:24.084 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.082+0000 7fee448bee00 1 -- 172.21.15.96:0/1917219493 shutdown_connections 2024-03-21T06:38:24.084 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.082+0000 7fee448bee00 1 -- 172.21.15.96:0/1917219493 wait complete. 2024-03-21T06:38:24.084 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.082+0000 7fee448bee00 1 Processor -- start 2024-03-21T06:38:24.084 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.082+0000 7fee448bee00 1 -- start start 2024-03-21T06:38:24.087 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.083+0000 7fee448bee00 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f1a4c5640 0x556f1a5449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:38:24.087 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.083+0000 7fee448bee00 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1a400550 con 0x556f1a4c5640 2024-03-21T06:38:24.087 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.083+0000 7fee3d99a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f1a4c5640 0x556f1a5449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:38:24.087 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.083+0000 7fee3d99a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f1a4c5640 0x556f1a5449c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55826/0 (socket says 172.21.15.96:55826) 2024-03-21T06:38:24.087 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.083+0000 7fee3d99a640 1 -- 172.21.15.96:0/3135427677 learned_addr learned my addr 172.21.15.96:0/3135427677 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:38:24.087 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.084+0000 7fee3d99a640 1 -- 172.21.15.96:0/3135427677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f1a3f5680 con 0x556f1a4c5640 2024-03-21T06:38:24.087 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.084+0000 7fee3d99a640 1 --2- 172.21.15.96:0/3135427677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f1a4c5640 0x556f1a5449c0 crc :-1 s=READY pgs=12966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:38:24.087 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.084+0000 7fee2effd640 1 -- 172.21.15.96:0/3135427677 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fee34039040 con 0x556f1a4c5640 2024-03-21T06:38:24.087 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.084+0000 7fee2effd640 1 -- 172.21.15.96:0/3135427677 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fee34033cc0 con 0x556f1a4c5640 2024-03-21T06:38:24.087 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.084+0000 7fee448bee00 1 -- 172.21.15.96:0/3135427677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556f1a3f50e0 con 0x556f1a4c5640 2024-03-21T06:38:24.087 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.084+0000 7fee2effd640 1 -- 172.21.15.96:0/3135427677 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fee34029ea0 con 0x556f1a4c5640 2024-03-21T06:38:24.087 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.084+0000 7fee448bee00 1 -- 172.21.15.96:0/3135427677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f1a548a40 con 0x556f1a4c5640 2024-03-21T06:38:24.087 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.085+0000 7fee2effd640 1 -- 172.21.15.96:0/3135427677 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fee3403c040 con 0x556f1a4c5640 2024-03-21T06:38:24.088 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.085+0000 7fee2effd640 1 --2- 172.21.15.96:0/3135427677 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fee1c036b30 0x7fee1c039040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:38:24.088 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.085+0000 7fee2effd640 1 -- 172.21.15.96:0/3135427677 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(707..707 src has 402..707) v4 ==== 4179+0+0 (crc 0 0 0) 0x7fee3402e040 con 0x556f1a4c5640 2024-03-21T06:38:24.088 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.085+0000 7fee2effd640 1 -- 172.21.15.96:0/3135427677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=708}) v3 -- 0x7fee1c03d4e0 con 0x556f1a4c5640 2024-03-21T06:38:24.088 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.085+0000 7fee448bee00 1 --2- 172.21.15.96:0/3135427677 >> v2:172.21.15.184:6804/155456265 conn(0x556f1a4c8ae0 0x556f1a5f1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:38:24.088 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.085+0000 7fee448bee00 1 -- 172.21.15.96:0/3135427677 --> v2:172.21.15.184:6804/155456265 -- osd_op(unknown.0.0:1 200.2 200:47cad802:::threemore:head [writefull 0~11812 in=11812b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e707) v9 -- 0x556f1a5484f0 con 0x556f1a4c8ae0 2024-03-21T06:38:24.088 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.085+0000 7fee3d199640 1 --2- 172.21.15.96:0/3135427677 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fee1c036b30 0x7fee1c039040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:38:24.088 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.086+0000 7fee3e19b640 1 --2- 172.21.15.96:0/3135427677 >> v2:172.21.15.184:6804/155456265 conn(0x556f1a4c8ae0 0x556f1a5f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:38:24.088 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.086+0000 7fee3d199640 1 --2- 172.21.15.96:0/3135427677 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fee1c036b30 0x7fee1c039040 crc :-1 s=READY pgs=3857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:38:24.089 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.088+0000 7fee3e19b640 1 --2- 172.21.15.96:0/3135427677 >> v2:172.21.15.184:6804/155456265 conn(0x556f1a4c8ae0 0x556f1a5f1580 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:38:24.098 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.097+0000 7fee3e19b640 1 -- 172.21.15.96:0/3135427677 <== osd.2 v2:172.21.15.184:6804/155456265 1 ==== osd_op_reply(1 threemore [writefull 0~11812] v707'1 uv1 ondisk = 0) v8 ==== 153+0+0 (crc 0 0 0) 0x7fee3000c430 con 0x556f1a4c8ae0 2024-03-21T06:38:24.098 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.097+0000 7fee448bee00 1 -- 172.21.15.96:0/3135427677 >> v2:172.21.15.184:6804/155456265 conn(0x556f1a4c8ae0 msgr2=0x556f1a5f1580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:24.099 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.097+0000 7fee448bee00 1 --2- 172.21.15.96:0/3135427677 >> v2:172.21.15.184:6804/155456265 conn(0x556f1a4c8ae0 0x556f1a5f1580 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:24.099 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.097+0000 7fee448bee00 1 -- 172.21.15.96:0/3135427677 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fee1c036b30 msgr2=0x7fee1c039040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:24.099 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.097+0000 7fee448bee00 1 --2- 172.21.15.96:0/3135427677 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fee1c036b30 0x7fee1c039040 crc :-1 s=READY pgs=3857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:24.099 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.097+0000 7fee448bee00 1 -- 172.21.15.96:0/3135427677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f1a4c5640 msgr2=0x556f1a5449c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:24.099 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.097+0000 7fee448bee00 1 --2- 172.21.15.96:0/3135427677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f1a4c5640 0x556f1a5449c0 crc :-1 s=READY pgs=12966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:24.099 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.097+0000 7fee448bee00 1 -- 172.21.15.96:0/3135427677 shutdown_connections 2024-03-21T06:38:24.099 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.097+0000 7fee448bee00 1 --2- 172.21.15.96:0/3135427677 >> v2:172.21.15.184:6804/155456265 conn(0x556f1a4c8ae0 0x556f1a5f1580 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:24.099 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.097+0000 7fee448bee00 1 --2- 172.21.15.96:0/3135427677 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fee1c036b30 0x7fee1c039040 unknown :-1 s=CLOSED pgs=3857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:24.099 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.097+0000 7fee448bee00 1 --2- 172.21.15.96:0/3135427677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f1a4c5640 0x556f1a5449c0 unknown :-1 s=CLOSED pgs=12966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:24.099 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.098+0000 7fee448bee00 1 -- 172.21.15.96:0/3135427677 >> 172.21.15.96:0/3135427677 conn(0x556f1a3d4e00 msgr2=0x556f1a4bff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:38:24.099 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.098+0000 7fee448bee00 1 -- 172.21.15.96:0/3135427677 shutdown_connections 2024-03-21T06:38:24.099 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.098+0000 7fee448bee00 1 -- 172.21.15.96:0/3135427677 wait complete. 2024-03-21T06:38:24.100 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph osd pool set-quota 13dbd6ab-5566-4934-957c-dcb5f3ce8145 max_bytes 0 2024-03-21T06:38:24.191 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.183+0000 7fe9a5353640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:38:24.192 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.193+0000 7fe9a5353640 1 Processor -- start 2024-03-21T06:38:24.192 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.193+0000 7fe9a5353640 1 -- start start 2024-03-21T06:38:24.193 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.194+0000 7fe9a5353640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe9a0134340 0x7fe9a0136760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:38:24.193 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.194+0000 7fe9a5353640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a009d6b0 con 0x7fe9a0131580 2024-03-21T06:38:24.193 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.194+0000 7fe9a5353640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a009cb00 con 0x7fe9a0134340 2024-03-21T06:38:24.193 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.194+0000 7fe997fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe9a0134340 0x7fe9a0136760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:38:24.194 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.194+0000 7fe99effd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fe9a0131580 0x7fe9a0131990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:53040/0 (socket says 172.21.15.96:53040) 2024-03-21T06:38:24.194 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.194+0000 7fe99effd640 1 -- 172.21.15.96:0/2307684397 learned_addr learned my addr 172.21.15.96:0/2307684397 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:38:24.194 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.194+0000 7fe997fff640 1 -- 172.21.15.96:0/2307684397 >> v1:172.21.15.96:6789/0 conn(0x7fe9a0131580 legacy=0x7fe9a0131990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:24.194 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.194+0000 7fe997fff640 1 -- 172.21.15.96:0/2307684397 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a00a4760 con 0x7fe9a0134340 2024-03-21T06:38:24.194 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.194+0000 7fe997fff640 1 --2- 172.21.15.96:0/2307684397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9a0134340 0x7fe9a0136760 crc :-1 s=READY pgs=12968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a7f14a5f069dea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:38:24.194 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.194+0000 7fe99e7fc640 1 -- 172.21.15.96:0/2307684397 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe988009ce0 con 0x7fe9a0134340 2024-03-21T06:38:24.194 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.194+0000 7fe99e7fc640 1 -- 172.21.15.96:0/2307684397 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe988009e40 con 0x7fe9a0134340 2024-03-21T06:38:24.194 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.194+0000 7fe99e7fc640 1 -- 172.21.15.96:0/2307684397 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe98800e360 con 0x7fe9a0134340 2024-03-21T06:38:24.195 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.195+0000 7fe9a5353640 1 -- 172.21.15.96:0/2307684397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9a0134340 msgr2=0x7fe9a0136760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:24.195 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.195+0000 7fe9a5353640 1 --2- 172.21.15.96:0/2307684397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9a0134340 0x7fe9a0136760 crc :-1 s=READY pgs=12968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:24.195 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.195+0000 7fe9a5353640 1 -- 172.21.15.96:0/2307684397 shutdown_connections 2024-03-21T06:38:24.196 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.195+0000 7fe9a5353640 1 --2- 172.21.15.96:0/2307684397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9a0134340 0x7fe9a0136760 unknown :-1 s=CLOSED pgs=12968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:24.196 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.195+0000 7fe9a5353640 1 -- 172.21.15.96:0/2307684397 >> 172.21.15.96:0/2307684397 conn(0x7fe9a009e2e0 msgr2=0x7fe9a009e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:38:24.196 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.195+0000 7fe9a5353640 1 -- 172.21.15.96:0/2307684397 shutdown_connections 2024-03-21T06:38:24.196 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.195+0000 7fe9a5353640 1 -- 172.21.15.96:0/2307684397 wait complete. 2024-03-21T06:38:24.196 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.195+0000 7fe9a5353640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:38:24.196 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.196+0000 7fe9a5353640 1 Processor -- start 2024-03-21T06:38:24.196 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.196+0000 7fe9a5353640 1 -- start start 2024-03-21T06:38:24.199 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.197+0000 7fe9a5353640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9a0131580 0x7fe9a011d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:38:24.199 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.197+0000 7fe9a5353640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a011efa0 con 0x7fe9a0131580 2024-03-21T06:38:24.199 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.197+0000 7fe99effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9a0131580 0x7fe9a011d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:38:24.199 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.197+0000 7fe99effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9a0131580 0x7fe9a011d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55848/0 (socket says 172.21.15.96:55848) 2024-03-21T06:38:24.199 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.197+0000 7fe99effd640 1 -- 172.21.15.96:0/1736080180 learned_addr learned my addr 172.21.15.96:0/1736080180 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:38:24.199 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.198+0000 7fe99effd640 1 -- 172.21.15.96:0/1736080180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a011ce50 con 0x7fe9a0131580 2024-03-21T06:38:24.199 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.198+0000 7fe99effd640 1 --2- 172.21.15.96:0/1736080180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9a0131580 0x7fe9a011d310 crc :-1 s=READY pgs=12969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:38:24.199 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.198+0000 7fe9977fe640 1 -- 172.21.15.96:0/1736080180 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe9900058a0 con 0x7fe9a0131580 2024-03-21T06:38:24.199 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.198+0000 7fe9977fe640 1 -- 172.21.15.96:0/1736080180 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe990005a00 con 0x7fe9a0131580 2024-03-21T06:38:24.200 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.198+0000 7fe9977fe640 1 -- 172.21.15.96:0/1736080180 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe990005ca0 con 0x7fe9a0131580 2024-03-21T06:38:24.200 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.198+0000 7fe9a5353640 1 -- 172.21.15.96:0/1736080180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a011f1d0 con 0x7fe9a0131580 2024-03-21T06:38:24.200 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.198+0000 7fe9a5353640 1 -- 172.21.15.96:0/1736080180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a012eea0 con 0x7fe9a0131580 2024-03-21T06:38:24.200 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.199+0000 7fe9977fe640 1 -- 172.21.15.96:0/1736080180 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fe9900070b0 con 0x7fe9a0131580 2024-03-21T06:38:24.200 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.199+0000 7fe9a5353640 1 -- 172.21.15.96:0/1736080180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a0068e80 con 0x7fe9a0131580 2024-03-21T06:38:24.200 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.199+0000 7fe9977fe640 1 --2- 172.21.15.96:0/1736080180 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fe980038260 0x7fe98003a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:38:24.200 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.199+0000 7fe9977fe640 1 -- 172.21.15.96:0/1736080180 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(707..707 src has 402..707) v4 ==== 4179+0+0 (crc 0 0 0) 0x7fe990047ed0 con 0x7fe9a0131580 2024-03-21T06:38:24.200 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.199+0000 7fe9977fe640 1 -- 172.21.15.96:0/1736080180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=708}) v3 -- 0x7fe98003ec10 con 0x7fe9a0131580 2024-03-21T06:38:24.200 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.199+0000 7fe997fff640 1 --2- 172.21.15.96:0/1736080180 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fe980038260 0x7fe98003a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:38:24.200 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.200+0000 7fe997fff640 1 --2- 172.21.15.96:0/1736080180 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fe980038260 0x7fe98003a770 crc :-1 s=READY pgs=3858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:38:24.203 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.204+0000 7fe9977fe640 1 -- 172.21.15.96:0/1736080180 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fe99000f230 con 0x7fe9a0131580 2024-03-21T06:38:24.378 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.379+0000 7fe9a5353640 1 -- 172.21.15.96:0/1736080180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fe9a00691d0 con 0x7fe9a0131580 2024-03-21T06:38:24.475 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.473+0000 7fe9977fe640 1 -- 172.21.15.96:0/1736080180 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 v708) v1 ==== 217+0+0 (crc 0 0 0) 0x7fe99000d660 con 0x7fe9a0131580 2024-03-21T06:38:24.475 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.474+0000 7fe9977fe640 1 -- 172.21.15.96:0/1736080180 <== mon.0 v2:172.21.15.96:3300/0 8 ==== osd_map(708..708 src has 402..708) v4 ==== 628+0+0 (crc 0 0 0) 0x7fe99000dca0 con 0x7fe9a0131580 2024-03-21T06:38:24.475 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.474+0000 7fe9977fe640 1 -- 172.21.15.96:0/1736080180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=709}) v3 -- 0x7fe98003fc00 con 0x7fe9a0131580 2024-03-21T06:38:24.577 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:24.578+0000 7fe9a5353640 1 -- 172.21.15.96:0/1736080180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fe9a0120380 con 0x7fe9a0131580 2024-03-21T06:38:25.476 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.477+0000 7fe9977fe640 1 -- 172.21.15.96:0/1736080180 <== mon.0 v2:172.21.15.96:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 v709) v1 ==== 217+0+0 (crc 0 0 0) 0x7fe99000d840 con 0x7fe9a0131580 2024-03-21T06:38:25.476 INFO:tasks.workunit.client.0.smithi096.stderr:set-quota max_bytes = 0 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 2024-03-21T06:38:25.476 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.477+0000 7fe9977fe640 1 -- 172.21.15.96:0/1736080180 <== mon.0 v2:172.21.15.96:3300/0 10 ==== osd_map(709..709 src has 402..709) v4 ==== 628+0+0 (crc 0 0 0) 0x7fe990046b20 con 0x7fe9a0131580 2024-03-21T06:38:25.477 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.477+0000 7fe9977fe640 1 -- 172.21.15.96:0/1736080180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=710}) v3 -- 0x7fe98003fd20 con 0x7fe9a0131580 2024-03-21T06:38:25.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.479+0000 7fe9a5353640 1 -- 172.21.15.96:0/1736080180 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fe980038260 msgr2=0x7fe98003a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:25.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.479+0000 7fe9a5353640 1 --2- 172.21.15.96:0/1736080180 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fe980038260 0x7fe98003a770 crc :-1 s=READY pgs=3858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:25.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.479+0000 7fe9a5353640 1 -- 172.21.15.96:0/1736080180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9a0131580 msgr2=0x7fe9a011d310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:25.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.479+0000 7fe9a5353640 1 --2- 172.21.15.96:0/1736080180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9a0131580 0x7fe9a011d310 crc :-1 s=READY pgs=12969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:25.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.479+0000 7fe9a5353640 1 -- 172.21.15.96:0/1736080180 shutdown_connections 2024-03-21T06:38:25.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.479+0000 7fe9a5353640 1 --2- 172.21.15.96:0/1736080180 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fe980038260 0x7fe98003a770 unknown :-1 s=CLOSED pgs=3858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:25.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.479+0000 7fe9a5353640 1 --2- 172.21.15.96:0/1736080180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9a0131580 0x7fe9a011d310 unknown :-1 s=CLOSED pgs=12969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:25.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.479+0000 7fe9a5353640 1 -- 172.21.15.96:0/1736080180 >> 172.21.15.96:0/1736080180 conn(0x7fe9a009e2e0 msgr2=0x7fe9a006c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:38:25.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.479+0000 7fe9a5353640 1 -- 172.21.15.96:0/1736080180 shutdown_connections 2024-03-21T06:38:25.479 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.479+0000 7fe9a5353640 1 -- 172.21.15.96:0/1736080180 wait complete. 2024-03-21T06:38:25.490 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph osd pool set-quota 13dbd6ab-5566-4934-957c-dcb5f3ce8145 max_objects 0 2024-03-21T06:38:25.588 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.581+0000 7f49e9b5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:38:25.588 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.589+0000 7f49e9b5c640 1 Processor -- start 2024-03-21T06:38:25.588 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.590+0000 7f49e9b5c640 1 -- start start 2024-03-21T06:38:25.590 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.590+0000 7f49e9b5c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f49e40a2ce0 0x7f49e409fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:38:25.590 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.590+0000 7f49e9b5c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e40a3240 con 0x7f49e40a0380 2024-03-21T06:38:25.590 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.590+0000 7f49e9b5c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e406e660 con 0x7f49e40a2ce0 2024-03-21T06:38:25.590 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.590+0000 7f49e37fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f49e40a2ce0 0x7f49e409fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:38:25.591 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.590+0000 7f49e2ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f49e40a0380 0x7f49e4134290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:53056/0 (socket says 172.21.15.96:53056) 2024-03-21T06:38:25.591 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.590+0000 7f49e2ffd640 1 -- 172.21.15.96:0/578305362 learned_addr learned my addr 172.21.15.96:0/578305362 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:38:25.591 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.591+0000 7f49e37fe640 1 -- 172.21.15.96:0/578305362 >> v1:172.21.15.96:6789/0 conn(0x7f49e40a0380 legacy=0x7f49e4134290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:25.591 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.591+0000 7f49e37fe640 1 -- 172.21.15.96:0/578305362 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e406e240 con 0x7f49e40a2ce0 2024-03-21T06:38:25.591 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.591+0000 7f49e27fc640 1 -- 172.21.15.96:0/578305362 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713874162 0 0) 0x7f49e40a3240 con 0x7f49e40a0380 2024-03-21T06:38:25.591 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.591+0000 7f49e37fe640 1 --2- 172.21.15.96:0/578305362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e40a2ce0 0x7f49e409fe40 crc :-1 s=READY pgs=12971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4755baeb7d6fda58 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:38:25.591 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.591+0000 7f49e27fc640 1 -- 172.21.15.96:0/578305362 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f49d8009ce0 con 0x7f49e40a2ce0 2024-03-21T06:38:25.591 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.591+0000 7f49e27fc640 1 -- 172.21.15.96:0/578305362 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f49d8009e40 con 0x7f49e40a2ce0 2024-03-21T06:38:25.591 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.591+0000 7f49e27fc640 1 -- 172.21.15.96:0/578305362 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f49d800e360 con 0x7f49e40a2ce0 2024-03-21T06:38:25.592 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.592+0000 7f49e9b5c640 1 -- 172.21.15.96:0/578305362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e40a2ce0 msgr2=0x7f49e409fe40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:25.592 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.592+0000 7f49e9b5c640 1 --2- 172.21.15.96:0/578305362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e40a2ce0 0x7f49e409fe40 crc :-1 s=READY pgs=12971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:25.593 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.592+0000 7f49e9b5c640 1 -- 172.21.15.96:0/578305362 shutdown_connections 2024-03-21T06:38:25.593 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.592+0000 7f49e9b5c640 1 --2- 172.21.15.96:0/578305362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e40a2ce0 0x7f49e409fe40 unknown :-1 s=CLOSED pgs=12971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:25.593 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.592+0000 7f49e9b5c640 1 -- 172.21.15.96:0/578305362 >> 172.21.15.96:0/578305362 conn(0x7f49e406ca30 msgr2=0x7f49e406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:38:25.593 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.592+0000 7f49e9b5c640 1 -- 172.21.15.96:0/578305362 shutdown_connections 2024-03-21T06:38:25.593 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.592+0000 7f49e9b5c640 1 -- 172.21.15.96:0/578305362 wait complete. 2024-03-21T06:38:25.593 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.592+0000 7f49e9b5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:38:25.593 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.593+0000 7f49e9b5c640 1 Processor -- start 2024-03-21T06:38:25.593 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.594+0000 7f49e9b5c640 1 -- start start 2024-03-21T06:38:25.596 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.594+0000 7f49e9b5c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e40a0380 0x7f49e4123c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:38:25.597 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.594+0000 7f49e9b5c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e4127e40 con 0x7f49e40a0380 2024-03-21T06:38:25.597 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.594+0000 7f49e37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e40a0380 0x7f49e4123c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:38:25.597 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.594+0000 7f49e37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e40a0380 0x7f49e4123c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55868/0 (socket says 172.21.15.96:55868) 2024-03-21T06:38:25.597 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.594+0000 7f49e37fe640 1 -- 172.21.15.96:0/2248513082 learned_addr learned my addr 172.21.15.96:0/2248513082 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:38:25.597 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.594+0000 7f49e37fe640 1 -- 172.21.15.96:0/2248513082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e4125100 con 0x7f49e40a0380 2024-03-21T06:38:25.597 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.595+0000 7f49e37fe640 1 --2- 172.21.15.96:0/2248513082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e40a0380 0x7f49e4123c40 crc :-1 s=READY pgs=12972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:38:25.597 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.595+0000 7f49cbfff640 1 -- 172.21.15.96:0/2248513082 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f49d8014540 con 0x7f49e40a0380 2024-03-21T06:38:25.597 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.595+0000 7f49cbfff640 1 -- 172.21.15.96:0/2248513082 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f49d80146a0 con 0x7f49e40a0380 2024-03-21T06:38:25.597 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.595+0000 7f49e9b5c640 1 -- 172.21.15.96:0/2248513082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e4124d60 con 0x7f49e40a0380 2024-03-21T06:38:25.597 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.595+0000 7f49cbfff640 1 -- 172.21.15.96:0/2248513082 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f49d8014940 con 0x7f49e40a0380 2024-03-21T06:38:25.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.595+0000 7f49e9b5c640 1 -- 172.21.15.96:0/2248513082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e41267e0 con 0x7f49e40a0380 2024-03-21T06:38:25.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.596+0000 7f49cbfff640 1 -- 172.21.15.96:0/2248513082 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f49d8014af0 con 0x7f49e40a0380 2024-03-21T06:38:25.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.596+0000 7f49e9b5c640 1 -- 172.21.15.96:0/2248513082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e41264f0 con 0x7f49e40a0380 2024-03-21T06:38:25.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.596+0000 7f49cbfff640 1 --2- 172.21.15.96:0/2248513082 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f49d0038260 0x7f49d003a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:38:25.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.596+0000 7f49cbfff640 1 -- 172.21.15.96:0/2248513082 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(709..709 src has 402..709) v4 ==== 4179+0+0 (crc 0 0 0) 0x7f49d8019040 con 0x7f49e40a0380 2024-03-21T06:38:25.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.596+0000 7f49cbfff640 1 -- 172.21.15.96:0/2248513082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=710}) v3 -- 0x7f49d003ec10 con 0x7f49e40a0380 2024-03-21T06:38:25.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.596+0000 7f49e2ffd640 1 --2- 172.21.15.96:0/2248513082 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f49d0038260 0x7f49d003a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:38:25.598 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.597+0000 7f49e2ffd640 1 --2- 172.21.15.96:0/2248513082 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f49d0038260 0x7f49d003a770 crc :-1 s=READY pgs=3859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:38:25.600 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.601+0000 7f49cbfff640 1 -- 172.21.15.96:0/2248513082 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f49d800f7b0 con 0x7f49e40a0380 2024-03-21T06:38:25.772 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:25.773+0000 7f49e9b5c640 1 -- 172.21.15.96:0/2248513082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f49e41250e0 con 0x7f49e40a0380 2024-03-21T06:38:26.476 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:26.477+0000 7f49cbfff640 1 -- 172.21.15.96:0/2248513082 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 v710) v1 ==== 221+0+0 (crc 0 0 0) 0x7f49d800f7b0 con 0x7f49e40a0380 2024-03-21T06:38:26.477 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:26.478+0000 7f49cbfff640 1 -- 172.21.15.96:0/2248513082 <== mon.0 v2:172.21.15.96:3300/0 8 ==== osd_map(710..710 src has 402..710) v4 ==== 628+0+0 (crc 0 0 0) 0x7f49d800f990 con 0x7f49e40a0380 2024-03-21T06:38:26.477 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:26.478+0000 7f49cbfff640 1 -- 172.21.15.96:0/2248513082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=711}) v3 -- 0x7f49d003fc70 con 0x7f49e40a0380 2024-03-21T06:38:26.581 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:26.582+0000 7f49e9b5c640 1 -- 172.21.15.96:0/2248513082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f49e4068e80 con 0x7f49e40a0380 2024-03-21T06:38:27.462 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:27.463+0000 7f49cbfff640 1 -- 172.21.15.96:0/2248513082 <== mon.0 v2:172.21.15.96:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 v711) v1 ==== 221+0+0 (crc 0 0 0) 0x7f49d800fba0 con 0x7f49e40a0380 2024-03-21T06:38:27.462 INFO:tasks.workunit.client.0.smithi096.stderr:set-quota max_objects = 0 for pool 13dbd6ab-5566-4934-957c-dcb5f3ce8145 2024-03-21T06:38:27.462 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:27.463+0000 7f49cbfff640 1 -- 172.21.15.96:0/2248513082 <== mon.0 v2:172.21.15.96:3300/0 10 ==== osd_map(711..711 src has 402..711) v4 ==== 628+0+0 (crc 0 0 0) 0x7f49d800fba0 con 0x7f49e40a0380 2024-03-21T06:38:27.463 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:27.463+0000 7f49cbfff640 1 -- 172.21.15.96:0/2248513082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=712}) v3 -- 0x7f49d003feb0 con 0x7f49e40a0380 2024-03-21T06:38:27.465 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:27.466+0000 7f49e9b5c640 1 -- 172.21.15.96:0/2248513082 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f49d0038260 msgr2=0x7f49d003a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:27.465 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:27.466+0000 7f49e9b5c640 1 --2- 172.21.15.96:0/2248513082 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f49d0038260 0x7f49d003a770 crc :-1 s=READY pgs=3859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:27.465 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:27.466+0000 7f49e9b5c640 1 -- 172.21.15.96:0/2248513082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e40a0380 msgr2=0x7f49e4123c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:27.466 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:27.466+0000 7f49e9b5c640 1 --2- 172.21.15.96:0/2248513082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e40a0380 0x7f49e4123c40 crc :-1 s=READY pgs=12972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:27.466 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:27.466+0000 7f49e9b5c640 1 -- 172.21.15.96:0/2248513082 shutdown_connections 2024-03-21T06:38:27.466 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:27.466+0000 7f49e9b5c640 1 --2- 172.21.15.96:0/2248513082 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f49d0038260 0x7f49d003a770 unknown :-1 s=CLOSED pgs=3859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:27.466 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:27.466+0000 7f49e9b5c640 1 --2- 172.21.15.96:0/2248513082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e40a0380 0x7f49e4123c40 unknown :-1 s=CLOSED pgs=12972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:27.466 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:27.466+0000 7f49e9b5c640 1 -- 172.21.15.96:0/2248513082 >> 172.21.15.96:0/2248513082 conn(0x7f49e406ca30 msgr2=0x7f49e411d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:38:27.466 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:27.466+0000 7f49e9b5c640 1 -- 172.21.15.96:0/2248513082 shutdown_connections 2024-03-21T06:38:27.466 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:27.466+0000 7f49e9b5c640 1 -- 172.21.15.96:0/2248513082 wait complete. 2024-03-21T06:38:27.477 INFO:tasks.workunit.client.0.smithi096.stderr:+ sleep 30 2024-03-21T06:38:57.479 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph osd pool delete 13dbd6ab-5566-4934-957c-dcb5f3ce8145 13dbd6ab-5566-4934-957c-dcb5f3ce8145 --yes-i-really-really-mean-it 2024-03-21T06:38:57.576 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.569+0000 7f6d60d9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:38:57.585 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.578+0000 7f6d60d9c640 1 Processor -- start 2024-03-21T06:38:57.585 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.578+0000 7f6d60d9c640 1 -- start start 2024-03-21T06:38:57.585 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.579+0000 7f6d60d9c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6d5c1325e0 0x7f6d5c1329f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:38:57.585 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.579+0000 7f6d60d9c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d5c131f80 con 0x7f6d5c09ca30 2024-03-21T06:38:57.586 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.579+0000 7f6d60d9c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5c132f30 con 0x7f6d5c1325e0 2024-03-21T06:38:57.586 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.579+0000 7f6d5a59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6d5c1325e0 0x7f6d5c1329f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:38:57.586 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.579+0000 7f6d53fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f6d5c09ca30 0x7f6d5c09ce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:53064/0 (socket says 172.21.15.96:53064) 2024-03-21T06:38:57.586 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.579+0000 7f6d53fff640 1 -- 172.21.15.96:0/1974711839 learned_addr learned my addr 172.21.15.96:0/1974711839 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:38:57.586 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.579+0000 7f6d5a59c640 1 -- 172.21.15.96:0/1974711839 >> v1:172.21.15.96:6789/0 conn(0x7f6d5c09ca30 legacy=0x7f6d5c09ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:57.587 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.579+0000 7f6d5a59c640 1 -- 172.21.15.96:0/1974711839 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c067f30 con 0x7f6d5c1325e0 2024-03-21T06:38:57.587 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.579+0000 7f6d59d9b640 1 -- 172.21.15.96:0/1974711839 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889834979 0 0) 0x7f6d5c131f80 con 0x7f6d5c09ca30 2024-03-21T06:38:57.587 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.579+0000 7f6d5a59c640 1 --2- 172.21.15.96:0/1974711839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d5c1325e0 0x7f6d5c1329f0 crc :-1 s=READY pgs=12974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa63c6084f301247 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:38:57.587 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.579+0000 7f6d59d9b640 1 -- 172.21.15.96:0/1974711839 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6d4c009ce0 con 0x7f6d5c1325e0 2024-03-21T06:38:57.587 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.579+0000 7f6d59d9b640 1 -- 172.21.15.96:0/1974711839 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f6d4c009e40 con 0x7f6d5c1325e0 2024-03-21T06:38:57.587 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.579+0000 7f6d59d9b640 1 -- 172.21.15.96:0/1974711839 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6d4c00e360 con 0x7f6d5c1325e0 2024-03-21T06:38:57.588 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.580+0000 7f6d60d9c640 1 -- 172.21.15.96:0/1974711839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d5c1325e0 msgr2=0x7f6d5c1329f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:57.588 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.580+0000 7f6d60d9c640 1 --2- 172.21.15.96:0/1974711839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d5c1325e0 0x7f6d5c1329f0 crc :-1 s=READY pgs=12974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:57.588 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.580+0000 7f6d60d9c640 1 -- 172.21.15.96:0/1974711839 shutdown_connections 2024-03-21T06:38:57.588 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.580+0000 7f6d60d9c640 1 --2- 172.21.15.96:0/1974711839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d5c1325e0 0x7f6d5c1329f0 unknown :-1 s=CLOSED pgs=12974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:57.588 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.580+0000 7f6d60d9c640 1 -- 172.21.15.96:0/1974711839 >> 172.21.15.96:0/1974711839 conn(0x7f6d5c0a05c0 msgr2=0x7f6d5c0a09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:38:57.588 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.580+0000 7f6d60d9c640 1 -- 172.21.15.96:0/1974711839 shutdown_connections 2024-03-21T06:38:57.589 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.580+0000 7f6d60d9c640 1 -- 172.21.15.96:0/1974711839 wait complete. 2024-03-21T06:38:57.589 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.580+0000 7f6d60d9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:38:57.589 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.581+0000 7f6d60d9c640 1 Processor -- start 2024-03-21T06:38:57.589 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.581+0000 7f6d60d9c640 1 -- start start 2024-03-21T06:38:57.589 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.582+0000 7f6d60d9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d5c09ca30 0x7f6d5c06dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:38:57.589 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.582+0000 7f6d60d9c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c06efa0 con 0x7f6d5c09ca30 2024-03-21T06:38:57.590 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.582+0000 7f6d5a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d5c09ca30 0x7f6d5c06dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:38:57.590 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.582+0000 7f6d5a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d5c09ca30 0x7f6d5c06dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55824/0 (socket says 172.21.15.96:55824) 2024-03-21T06:38:57.590 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.582+0000 7f6d5a59c640 1 -- 172.21.15.96:0/2277845207 learned_addr learned my addr 172.21.15.96:0/2277845207 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:38:57.590 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.582+0000 7f6d5a59c640 1 -- 172.21.15.96:0/2277845207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c122f60 con 0x7f6d5c09ca30 2024-03-21T06:38:57.590 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.583+0000 7f6d5a59c640 1 --2- 172.21.15.96:0/2277845207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d5c09ca30 0x7f6d5c06dae0 crc :-1 s=READY pgs=12975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:38:57.590 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.583+0000 7f6d52ffd640 1 -- 172.21.15.96:0/2277845207 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6d4c014620 con 0x7f6d5c09ca30 2024-03-21T06:38:57.591 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.583+0000 7f6d52ffd640 1 -- 172.21.15.96:0/2277845207 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f6d4c014780 con 0x7f6d5c09ca30 2024-03-21T06:38:57.591 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.583+0000 7f6d60d9c640 1 -- 172.21.15.96:0/2277845207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5c06f6f0 con 0x7f6d5c09ca30 2024-03-21T06:38:57.591 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.583+0000 7f6d52ffd640 1 -- 172.21.15.96:0/2277845207 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6d4c014a20 con 0x7f6d5c09ca30 2024-03-21T06:38:57.591 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.583+0000 7f6d60d9c640 1 -- 172.21.15.96:0/2277845207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5c123990 con 0x7f6d5c09ca30 2024-03-21T06:38:57.591 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.584+0000 7f6d52ffd640 1 -- 172.21.15.96:0/2277845207 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f6d4c014bd0 con 0x7f6d5c09ca30 2024-03-21T06:38:57.591 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.584+0000 7f6d60d9c640 1 -- 172.21.15.96:0/2277845207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5c121df0 con 0x7f6d5c09ca30 2024-03-21T06:38:57.592 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.584+0000 7f6d52ffd640 1 --2- 172.21.15.96:0/2277845207 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f6d3c038260 0x7f6d3c03a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:38:57.592 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.584+0000 7f6d52ffd640 1 -- 172.21.15.96:0/2277845207 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(711..711 src has 402..711) v4 ==== 4179+0+0 (crc 0 0 0) 0x7f6d4c019040 con 0x7f6d5c09ca30 2024-03-21T06:38:57.592 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.584+0000 7f6d52ffd640 1 -- 172.21.15.96:0/2277845207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=712}) v3 -- 0x7f6d3c03ec10 con 0x7f6d5c09ca30 2024-03-21T06:38:57.592 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.584+0000 7f6d53fff640 1 --2- 172.21.15.96:0/2277845207 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f6d3c038260 0x7f6d3c03a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:38:57.592 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.585+0000 7f6d53fff640 1 --2- 172.21.15.96:0/2277845207 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f6d3c038260 0x7f6d3c03a770 crc :-1 s=READY pgs=3860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:38:57.593 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.589+0000 7f6d52ffd640 1 -- 172.21.15.96:0/2277845207 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f6d4c00eb00 con 0x7f6d5c09ca30 2024-03-21T06:38:57.762 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:57.763+0000 7f6d60d9c640 1 -- 172.21.15.96:0/2277845207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "pool2": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6d5c067f30 con 0x7f6d5c09ca30 2024-03-21T06:38:58.467 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.468+0000 7f6d52ffd640 1 -- 172.21.15.96:0/2277845207 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "pool2": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "yes_i_really_really_mean_it": true}]=0 pool '13dbd6ab-5566-4934-957c-dcb5f3ce8145' removed v712) v1 ==== 248+0+0 (crc 0 0 0) 0x7f6d5c067f30 con 0x7f6d5c09ca30 2024-03-21T06:38:58.468 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.469+0000 7f6d52ffd640 1 -- 172.21.15.96:0/2277845207 <== mon.0 v2:172.21.15.96:3300/0 8 ==== osd_map(712..712 src has 402..712) v4 ==== 296+0+0 (crc 0 0 0) 0x7f6d4c018630 con 0x7f6d5c09ca30 2024-03-21T06:38:58.468 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.469+0000 7f6d52ffd640 1 -- 172.21.15.96:0/2277845207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=713}) v3 -- 0x7f6d3c03f840 con 0x7f6d5c09ca30 2024-03-21T06:38:58.587 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.588+0000 7f6d60d9c640 1 -- 172.21.15.96:0/2277845207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "pool2": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6d5c067f30 con 0x7f6d5c09ca30 2024-03-21T06:38:58.587 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.588+0000 7f6d52ffd640 1 -- 172.21.15.96:0/2277845207 <== mon.0 v2:172.21.15.96:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "pool2": "13dbd6ab-5566-4934-957c-dcb5f3ce8145", "yes_i_really_really_mean_it": true}]=0 pool '13dbd6ab-5566-4934-957c-dcb5f3ce8145' does not exist v712) v1 ==== 255+0+0 (crc 0 0 0) 0x7f6d5c067f30 con 0x7f6d5c09ca30 2024-03-21T06:38:58.587 INFO:tasks.workunit.client.0.smithi096.stderr:pool '13dbd6ab-5566-4934-957c-dcb5f3ce8145' does not exist 2024-03-21T06:38:58.589 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.590+0000 7f6d60d9c640 1 -- 172.21.15.96:0/2277845207 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f6d3c038260 msgr2=0x7f6d3c03a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:58.589 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.590+0000 7f6d60d9c640 1 --2- 172.21.15.96:0/2277845207 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f6d3c038260 0x7f6d3c03a770 crc :-1 s=READY pgs=3860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:58.589 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.590+0000 7f6d60d9c640 1 -- 172.21.15.96:0/2277845207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d5c09ca30 msgr2=0x7f6d5c06dae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:58.590 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.590+0000 7f6d60d9c640 1 --2- 172.21.15.96:0/2277845207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d5c09ca30 0x7f6d5c06dae0 crc :-1 s=READY pgs=12975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:58.590 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.590+0000 7f6d60d9c640 1 -- 172.21.15.96:0/2277845207 shutdown_connections 2024-03-21T06:38:58.590 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.590+0000 7f6d60d9c640 1 --2- 172.21.15.96:0/2277845207 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f6d3c038260 0x7f6d3c03a770 unknown :-1 s=CLOSED pgs=3860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:58.590 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.590+0000 7f6d60d9c640 1 --2- 172.21.15.96:0/2277845207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d5c09ca30 0x7f6d5c06dae0 unknown :-1 s=CLOSED pgs=12975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:58.590 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.590+0000 7f6d60d9c640 1 -- 172.21.15.96:0/2277845207 >> 172.21.15.96:0/2277845207 conn(0x7f6d5c0a05c0 msgr2=0x7f6d5c1b9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:38:58.590 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.590+0000 7f6d60d9c640 1 -- 172.21.15.96:0/2277845207 shutdown_connections 2024-03-21T06:38:58.590 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.590+0000 7f6d60d9c640 1 -- 172.21.15.96:0/2277845207 wait complete. 2024-03-21T06:38:58.599 INFO:tasks.workunit.client.0.smithi096.stderr:+ ceph osd pool delete 416ac5b8-f35e-473b-b584-9498e7bbba57 416ac5b8-f35e-473b-b584-9498e7bbba57 --yes-i-really-really-mean-it 2024-03-21T06:38:58.687 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.679+0000 7efd1086d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:38:58.687 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.688+0000 7efd1086d640 1 Processor -- start 2024-03-21T06:38:58.687 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.688+0000 7efd1086d640 1 -- start start 2024-03-21T06:38:58.695 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.688+0000 7efd1086d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efd0c1314b0 0x7efd0c1318c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:38:58.696 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.688+0000 7efd1086d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0c130f70 con 0x7efd0c06dab0 2024-03-21T06:38:58.696 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.688+0000 7efd1086d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd0c131e00 con 0x7efd0c1314b0 2024-03-21T06:38:58.696 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.689+0000 7efd0a59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efd0c1314b0 0x7efd0c1318c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:38:58.696 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.689+0000 7efd09d9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7efd0c06dab0 0x7efd0c06dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:53080/0 (socket says 172.21.15.96:53080) 2024-03-21T06:38:58.696 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.689+0000 7efd09d9b640 1 -- 172.21.15.96:0/1683456229 learned_addr learned my addr 172.21.15.96:0/1683456229 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:38:58.696 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.689+0000 7efd0a59c640 1 -- 172.21.15.96:0/1683456229 >> v1:172.21.15.96:6789/0 conn(0x7efd0c06dab0 legacy=0x7efd0c06dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:58.696 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.689+0000 7efd0a59c640 1 -- 172.21.15.96:0/1683456229 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c1311a0 con 0x7efd0c1314b0 2024-03-21T06:38:58.696 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.689+0000 7efd0a59c640 1 --2- 172.21.15.96:0/1683456229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd0c1314b0 0x7efd0c1318c0 crc :-1 s=READY pgs=12977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a263006199d91db server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:38:58.697 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.689+0000 7efd0959a640 1 -- 172.21.15.96:0/1683456229 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efcfc009ce0 con 0x7efd0c1314b0 2024-03-21T06:38:58.697 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.689+0000 7efd0959a640 1 -- 172.21.15.96:0/1683456229 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7efcfc009e40 con 0x7efd0c1314b0 2024-03-21T06:38:58.697 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.689+0000 7efd0959a640 1 -- 172.21.15.96:0/1683456229 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efcfc00e360 con 0x7efd0c1314b0 2024-03-21T06:38:58.697 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.690+0000 7efd1086d640 1 -- 172.21.15.96:0/1683456229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd0c1314b0 msgr2=0x7efd0c1318c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:58.697 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.690+0000 7efd1086d640 1 --2- 172.21.15.96:0/1683456229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd0c1314b0 0x7efd0c1318c0 crc :-1 s=READY pgs=12977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:58.697 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.690+0000 7efd1086d640 1 -- 172.21.15.96:0/1683456229 shutdown_connections 2024-03-21T06:38:58.697 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.690+0000 7efd1086d640 1 --2- 172.21.15.96:0/1683456229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd0c1314b0 0x7efd0c1318c0 unknown :-1 s=CLOSED pgs=12977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:58.697 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.690+0000 7efd1086d640 1 -- 172.21.15.96:0/1683456229 >> 172.21.15.96:0/1683456229 conn(0x7efd0c0a05c0 msgr2=0x7efd0c0a09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:38:58.697 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.690+0000 7efd1086d640 1 -- 172.21.15.96:0/1683456229 shutdown_connections 2024-03-21T06:38:58.697 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.690+0000 7efd1086d640 1 -- 172.21.15.96:0/1683456229 wait complete. 2024-03-21T06:38:58.698 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.690+0000 7efd1086d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:38:58.698 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.691+0000 7efd1086d640 1 Processor -- start 2024-03-21T06:38:58.698 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.691+0000 7efd1086d640 1 -- start start 2024-03-21T06:38:58.698 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.692+0000 7efd1086d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd0c06dab0 0x7efd0c11ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:38:58.698 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.692+0000 7efd1086d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c06a530 con 0x7efd0c06dab0 2024-03-21T06:38:58.698 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.692+0000 7efd0a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd0c06dab0 0x7efd0c11ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:38:58.698 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.692+0000 7efd0a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd0c06dab0 0x7efd0c11ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55842/0 (socket says 172.21.15.96:55842) 2024-03-21T06:38:58.698 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.692+0000 7efd0a59c640 1 -- 172.21.15.96:0/2106038955 learned_addr learned my addr 172.21.15.96:0/2106038955 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:38:58.698 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.692+0000 7efd0a59c640 1 -- 172.21.15.96:0/2106038955 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c1246b0 con 0x7efd0c06dab0 2024-03-21T06:38:58.698 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.692+0000 7efd0a59c640 1 --2- 172.21.15.96:0/2106038955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd0c06dab0 0x7efd0c11ddd0 crc :-1 s=READY pgs=12978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:38:58.699 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.692+0000 7efcfaffd640 1 -- 172.21.15.96:0/2106038955 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efcfc014620 con 0x7efd0c06dab0 2024-03-21T06:38:58.699 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.693+0000 7efcfaffd640 1 -- 172.21.15.96:0/2106038955 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7efcfc014780 con 0x7efd0c06dab0 2024-03-21T06:38:58.699 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.693+0000 7efd1086d640 1 -- 172.21.15.96:0/2106038955 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd0c1243c0 con 0x7efd0c06dab0 2024-03-21T06:38:58.699 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.693+0000 7efcfaffd640 1 -- 172.21.15.96:0/2106038955 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7efcfc014a20 con 0x7efd0c06dab0 2024-03-21T06:38:58.699 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.693+0000 7efd1086d640 1 -- 172.21.15.96:0/2106038955 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0c11f290 con 0x7efd0c06dab0 2024-03-21T06:38:58.699 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.693+0000 7efcfaffd640 1 -- 172.21.15.96:0/2106038955 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7efcfc014bd0 con 0x7efd0c06dab0 2024-03-21T06:38:58.699 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.693+0000 7efd1086d640 1 -- 172.21.15.96:0/2106038955 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd0c068e80 con 0x7efd0c06dab0 2024-03-21T06:38:58.699 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.694+0000 7efcfaffd640 1 --2- 172.21.15.96:0/2106038955 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7efcec038210 0x7efcec03a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:38:58.699 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.694+0000 7efcfaffd640 1 -- 172.21.15.96:0/2106038955 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(712..712 src has 402..712) v4 ==== 3791+0+0 (crc 0 0 0) 0x7efcfc019040 con 0x7efd0c06dab0 2024-03-21T06:38:58.699 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.694+0000 7efcfaffd640 1 -- 172.21.15.96:0/2106038955 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=713}) v3 -- 0x7efcec03e420 con 0x7efd0c06dab0 2024-03-21T06:38:58.700 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.694+0000 7efd09d9b640 1 --2- 172.21.15.96:0/2106038955 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7efcec038210 0x7efcec03a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:38:58.700 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.694+0000 7efd09d9b640 1 --2- 172.21.15.96:0/2106038955 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7efcec038210 0x7efcec03a720 crc :-1 s=READY pgs=3861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:38:58.700 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.698+0000 7efcfaffd640 1 -- 172.21.15.96:0/2106038955 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7efcfc010310 con 0x7efd0c06dab0 2024-03-21T06:38:58.874 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:58.875+0000 7efd1086d640 1 -- 172.21.15.96:0/2106038955 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "416ac5b8-f35e-473b-b584-9498e7bbba57", "pool2": "416ac5b8-f35e-473b-b584-9498e7bbba57", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efd0c09b910 con 0x7efd0c06dab0 2024-03-21T06:38:59.476 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:59.477+0000 7efcfaffd640 1 -- 172.21.15.96:0/2106038955 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "416ac5b8-f35e-473b-b584-9498e7bbba57", "pool2": "416ac5b8-f35e-473b-b584-9498e7bbba57", "yes_i_really_really_mean_it": true}]=0 pool '416ac5b8-f35e-473b-b584-9498e7bbba57' removed v713) v1 ==== 248+0+0 (crc 0 0 0) 0x7efcfc018a20 con 0x7efd0c06dab0 2024-03-21T06:38:59.476 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:59.477+0000 7efcfaffd640 1 -- 172.21.15.96:0/2106038955 <== mon.0 v2:172.21.15.96:3300/0 8 ==== osd_map(713..713 src has 402..713) v4 ==== 296+0+0 (crc 0 0 0) 0x7efcfc00e230 con 0x7efd0c06dab0 2024-03-21T06:38:59.476 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:59.477+0000 7efcfaffd640 1 -- 172.21.15.96:0/2106038955 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=714}) v3 -- 0x7efcec03f090 con 0x7efd0c06dab0 2024-03-21T06:38:59.586 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:59.586+0000 7efd1086d640 1 -- 172.21.15.96:0/2106038955 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "416ac5b8-f35e-473b-b584-9498e7bbba57", "pool2": "416ac5b8-f35e-473b-b584-9498e7bbba57", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efd0c06bd30 con 0x7efd0c06dab0 2024-03-21T06:38:59.586 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:59.587+0000 7efcfaffd640 1 -- 172.21.15.96:0/2106038955 <== mon.0 v2:172.21.15.96:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "416ac5b8-f35e-473b-b584-9498e7bbba57", "pool2": "416ac5b8-f35e-473b-b584-9498e7bbba57", "yes_i_really_really_mean_it": true}]=0 pool '416ac5b8-f35e-473b-b584-9498e7bbba57' does not exist v713) v1 ==== 255+0+0 (crc 0 0 0) 0x7efcfc018c00 con 0x7efd0c06dab0 2024-03-21T06:38:59.586 INFO:tasks.workunit.client.0.smithi096.stderr:pool '416ac5b8-f35e-473b-b584-9498e7bbba57' does not exist 2024-03-21T06:38:59.588 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:59.589+0000 7efd1086d640 1 -- 172.21.15.96:0/2106038955 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7efcec038210 msgr2=0x7efcec03a720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:59.588 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:59.589+0000 7efd1086d640 1 --2- 172.21.15.96:0/2106038955 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7efcec038210 0x7efcec03a720 crc :-1 s=READY pgs=3861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:59.588 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:59.589+0000 7efd1086d640 1 -- 172.21.15.96:0/2106038955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd0c06dab0 msgr2=0x7efd0c11ddd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:38:59.589 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:59.589+0000 7efd1086d640 1 --2- 172.21.15.96:0/2106038955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd0c06dab0 0x7efd0c11ddd0 crc :-1 s=READY pgs=12978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:59.589 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:59.589+0000 7efd1086d640 1 -- 172.21.15.96:0/2106038955 shutdown_connections 2024-03-21T06:38:59.589 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:59.589+0000 7efd1086d640 1 --2- 172.21.15.96:0/2106038955 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7efcec038210 0x7efcec03a720 unknown :-1 s=CLOSED pgs=3861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:59.589 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:59.589+0000 7efd1086d640 1 --2- 172.21.15.96:0/2106038955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd0c06dab0 0x7efd0c11ddd0 unknown :-1 s=CLOSED pgs=12978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:38:59.589 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:59.589+0000 7efd1086d640 1 -- 172.21.15.96:0/2106038955 >> 172.21.15.96:0/2106038955 conn(0x7efd0c0a05c0 msgr2=0x7efd0c06fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:38:59.589 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:59.589+0000 7efd1086d640 1 -- 172.21.15.96:0/2106038955 shutdown_connections 2024-03-21T06:38:59.589 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-21T06:38:59.589+0000 7efd1086d640 1 -- 172.21.15.96:0/2106038955 wait complete. 2024-03-21T06:38:59.598 INFO:tasks.workunit.client.0.smithi096.stderr:+ echo OK 2024-03-21T06:38:59.599 INFO:tasks.workunit.client.0.smithi096.stdout:OK 2024-03-21T06:38:59.600 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-21T06:38:59.600 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T06:38:59.668 INFO:tasks.workunit:Stopping ['rados/test.sh --crimson', 'rados/test_pool_quota.sh'] on client.0... 2024-03-21T06:38:59.668 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-21T06:39:00.040 DEBUG:teuthology.parallel:result is None 2024-03-21T06:39:00.041 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T06:39:00.068 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T06:39:00.068 DEBUG:teuthology.orchestra.run.smithi096:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-21T06:39:00.144 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T06:39:00.144 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-21T06:39:00.158 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-21T06:39:00.159 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T06:39:00.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.287+0000 7f507366d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:00.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.291+0000 7f507366d640 1 Processor -- start 2024-03-21T06:39:00.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.291+0000 7f507366d640 1 -- start start 2024-03-21T06:39:00.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.292+0000 7f507366d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f506c099ed0 0x7f506c137470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:00.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.292+0000 7f507366d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f506c09ab40 con 0x7f506c0993b0 2024-03-21T06:39:00.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.292+0000 7f507366d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f506c09b390 con 0x7f506c099ed0 2024-03-21T06:39:00.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.292+0000 7f5070c08640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f506c099ed0 0x7f506c137470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:00.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.292+0000 7f5071409640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f506c0993b0 0x7f506c0997c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:53094/0 (socket says 172.21.15.96:53094) 2024-03-21T06:39:00.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.292+0000 7f5071409640 1 -- 172.21.15.96:0/2883518466 learned_addr learned my addr 172.21.15.96:0/2883518466 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:39:00.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.292+0000 7f5070c08640 1 -- 172.21.15.96:0/2883518466 >> v1:172.21.15.96:6789/0 conn(0x7f506c0993b0 legacy=0x7f506c0997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:00.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.292+0000 7f5070c08640 1 -- 172.21.15.96:0/2883518466 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c09af50 con 0x7f506c099ed0 2024-03-21T06:39:00.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.292+0000 7f5063fff640 1 -- 172.21.15.96:0/2883518466 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225053865 0 0) 0x7f506c09ab40 con 0x7f506c0993b0 2024-03-21T06:39:00.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.292+0000 7f5070c08640 1 --2- 172.21.15.96:0/2883518466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c099ed0 0x7f506c137470 crc :-1 s=READY pgs=12980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a0aaec2fe97622 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:00.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.293+0000 7f5063fff640 1 -- 172.21.15.96:0/2883518466 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f505c009ce0 con 0x7f506c099ed0 2024-03-21T06:39:00.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.293+0000 7f5063fff640 1 -- 172.21.15.96:0/2883518466 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f505c009e40 con 0x7f506c099ed0 2024-03-21T06:39:00.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.293+0000 7f5063fff640 1 -- 172.21.15.96:0/2883518466 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f505c00e360 con 0x7f506c099ed0 2024-03-21T06:39:00.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.293+0000 7f507366d640 1 -- 172.21.15.96:0/2883518466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c099ed0 msgr2=0x7f506c137470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:00.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.293+0000 7f507366d640 1 --2- 172.21.15.96:0/2883518466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c099ed0 0x7f506c137470 crc :-1 s=READY pgs=12980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:00.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.293+0000 7f507366d640 1 -- 172.21.15.96:0/2883518466 shutdown_connections 2024-03-21T06:39:00.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.293+0000 7f507366d640 1 --2- 172.21.15.96:0/2883518466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c099ed0 0x7f506c137470 unknown :-1 s=CLOSED pgs=12980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:00.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.293+0000 7f507366d640 1 -- 172.21.15.96:0/2883518466 >> 172.21.15.96:0/2883518466 conn(0x7f506c09fdb0 msgr2=0x7f506c0a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:00.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.293+0000 7f507366d640 1 -- 172.21.15.96:0/2883518466 shutdown_connections 2024-03-21T06:39:00.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.293+0000 7f507366d640 1 -- 172.21.15.96:0/2883518466 wait complete. 2024-03-21T06:39:00.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.294+0000 7f507366d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:00.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.295+0000 7f507366d640 1 Processor -- start 2024-03-21T06:39:00.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.295+0000 7f507366d640 1 -- start start 2024-03-21T06:39:00.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.295+0000 7f507366d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c0993b0 0x7f506c06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:00.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.295+0000 7f507366d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c1322d0 con 0x7f506c0993b0 2024-03-21T06:39:00.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.295+0000 7f5071409640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c0993b0 0x7f506c06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:00.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.295+0000 7f5071409640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c0993b0 0x7f506c06d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55862/0 (socket says 172.21.15.96:55862) 2024-03-21T06:39:00.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.295+0000 7f5071409640 1 -- 172.21.15.96:0/623719453 learned_addr learned my addr 172.21.15.96:0/623719453 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:39:00.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.296+0000 7f5071409640 1 -- 172.21.15.96:0/623719453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c0a1cd0 con 0x7f506c0993b0 2024-03-21T06:39:00.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.296+0000 7f5071409640 1 --2- 172.21.15.96:0/623719453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c0993b0 0x7f506c06d4f0 crc :-1 s=READY pgs=12981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:00.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.296+0000 7f5061ffb640 1 -- 172.21.15.96:0/623719453 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5068004af0 con 0x7f506c0993b0 2024-03-21T06:39:00.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.296+0000 7f5061ffb640 1 -- 172.21.15.96:0/623719453 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f5068004c50 con 0x7f506c0993b0 2024-03-21T06:39:00.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.296+0000 7f507366d640 1 -- 172.21.15.96:0/623719453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f506c131850 con 0x7f506c0993b0 2024-03-21T06:39:00.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.296+0000 7f5061ffb640 1 -- 172.21.15.96:0/623719453 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f5068007120 con 0x7f506c0993b0 2024-03-21T06:39:00.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.296+0000 7f507366d640 1 -- 172.21.15.96:0/623719453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f506c1302d0 con 0x7f506c0993b0 2024-03-21T06:39:00.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.297+0000 7f5061ffb640 1 -- 172.21.15.96:0/623719453 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f50680072d0 con 0x7f506c0993b0 2024-03-21T06:39:00.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.297+0000 7f507366d640 1 -- 172.21.15.96:0/623719453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f506c06f200 con 0x7f506c0993b0 2024-03-21T06:39:00.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.297+0000 7f5061ffb640 1 --2- 172.21.15.96:0/623719453 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5050038260 0x7f505003a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:00.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.297+0000 7f5061ffb640 1 -- 172.21.15.96:0/623719453 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(713..713 src has 402..713) v4 ==== 3403+0+0 (crc 0 0 0) 0x7f5068047c80 con 0x7f506c0993b0 2024-03-21T06:39:00.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.297+0000 7f5070c08640 1 --2- 172.21.15.96:0/623719453 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5050038260 0x7f505003a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:00.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.298+0000 7f5070c08640 1 --2- 172.21.15.96:0/623719453 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5050038260 0x7f505003a770 crc :-1 s=READY pgs=3862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:00.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.303+0000 7f5061ffb640 1 -- 172.21.15.96:0/623719453 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f506800a9e0 con 0x7f506c0993b0 2024-03-21T06:39:00.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.472+0000 7f507366d640 1 -- 172.21.15.96:0/623719453 --> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f506c09ba00 con 0x7f5050038260 2024-03-21T06:39:00.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.476+0000 7f5061ffb640 1 -- 172.21.15.96:0/623719453 <== mgr.4099 v2:172.21.15.184:6800/2376803682 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+107144 (crc 0 0 0) 0x7f505c0124e0 con 0x7f5050038260 2024-03-21T06:39:00.476 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:39:00.476 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-03-21T06:39:00.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.478+0000 7f507366d640 1 -- 172.21.15.96:0/623719453 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5050038260 msgr2=0x7f505003a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:00.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.478+0000 7f507366d640 1 --2- 172.21.15.96:0/623719453 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5050038260 0x7f505003a770 crc :-1 s=READY pgs=3862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:00.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.478+0000 7f507366d640 1 -- 172.21.15.96:0/623719453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c0993b0 msgr2=0x7f506c06d4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:00.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.478+0000 7f507366d640 1 --2- 172.21.15.96:0/623719453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c0993b0 0x7f506c06d4f0 crc :-1 s=READY pgs=12981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:00.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.478+0000 7f507366d640 1 -- 172.21.15.96:0/623719453 shutdown_connections 2024-03-21T06:39:00.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.478+0000 7f507366d640 1 --2- 172.21.15.96:0/623719453 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f5050038260 0x7f505003a770 unknown :-1 s=CLOSED pgs=3862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:00.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.478+0000 7f507366d640 1 --2- 172.21.15.96:0/623719453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c0993b0 0x7f506c06d4f0 unknown :-1 s=CLOSED pgs=12981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:00.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.478+0000 7f507366d640 1 -- 172.21.15.96:0/623719453 >> 172.21.15.96:0/623719453 conn(0x7f506c09fdb0 msgr2=0x7f506c1ba050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:00.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.478+0000 7f507366d640 1 -- 172.21.15.96:0/623719453 shutdown_connections 2024-03-21T06:39:00.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.478+0000 7f507366d640 1 -- 172.21.15.96:0/623719453 wait complete. 2024-03-21T06:39:00.488 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":1266,"stamp":"2024-03-21T06:38:59.478725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":288232,"kb_used_data":15240,"kb_used_omap":0,"kb_used_meta":272960,"kb_avail":419142168,"statfs":{"total":429496729600,"available":429201580032,"internally_reserved":0,"allocated":15605760,"data_stored":6956825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279511040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-295300,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-25,"num_write_kb":-300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.621107"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.760844+0000","last_change":"2024-03-21T06:25:58.485657+0000","last_active":"2024-03-21T06:38:29.760844+0000","last_peered":"2024-03-21T06:38:29.760844+0000","last_clean":"2024-03-21T06:38:29.760844+0000","last_became_active":"2024-03-21T06:25:58.480248+0000","last_became_peered":"2024-03-21T06:25:58.480248+0000","last_unstale":"2024-03-21T06:38:29.760844+0000","last_undegraded":"2024-03-21T06:38:29.760844+0000","last_fullsized":"2024-03-21T06:38:29.760844+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.500365+0000","last_change":"2024-03-21T06:25:58.492610+0000","last_active":"2024-03-21T06:38:58.500365+0000","last_peered":"2024-03-21T06:38:58.500365+0000","last_clean":"2024-03-21T06:38:58.500365+0000","last_became_active":"2024-03-21T06:25:58.484415+0000","last_became_peered":"2024-03-21T06:25:58.484415+0000","last_unstale":"2024-03-21T06:38:58.500365+0000","last_undegraded":"2024-03-21T06:38:58.500365+0000","last_fullsized":"2024-03-21T06:38:58.500365+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.521005+0000","last_change":"2024-03-21T06:25:58.492051+0000","last_active":"2024-03-21T06:38:27.521005+0000","last_peered":"2024-03-21T06:38:27.521005+0000","last_clean":"2024-03-21T06:38:27.521005+0000","last_became_active":"2024-03-21T06:25:58.487475+0000","last_became_peered":"2024-03-21T06:25:58.487475+0000","last_unstale":"2024-03-21T06:38:27.521005+0000","last_undegraded":"2024-03-21T06:38:27.521005+0000","last_fullsized":"2024-03-21T06:38:27.521005+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.713728+0000","last_change":"2024-03-21T06:25:58.528878+0000","last_active":"2024-03-21T06:38:29.713728+0000","last_peered":"2024-03-21T06:38:29.713728+0000","last_clean":"2024-03-21T06:38:29.713728+0000","last_became_active":"2024-03-21T06:25:58.519468+0000","last_became_peered":"2024-03-21T06:25:58.519468+0000","last_unstale":"2024-03-21T06:38:29.713728+0000","last_undegraded":"2024-03-21T06:38:29.713728+0000","last_fullsized":"2024-03-21T06:38:29.713728+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.747889+0000","last_change":"2024-03-21T06:25:58.530972+0000","last_active":"2024-03-21T06:38:29.747889+0000","last_peered":"2024-03-21T06:38:29.747889+0000","last_clean":"2024-03-21T06:38:29.747889+0000","last_became_active":"2024-03-21T06:25:58.527143+0000","last_became_peered":"2024-03-21T06:25:58.527143+0000","last_unstale":"2024-03-21T06:38:29.747889+0000","last_undegraded":"2024-03-21T06:38:29.747889+0000","last_fullsized":"2024-03-21T06:38:29.747889+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.483335+0000","last_change":"2024-03-21T06:25:58.506594+0000","last_active":"2024-03-21T06:38:58.483335+0000","last_peered":"2024-03-21T06:38:58.483335+0000","last_clean":"2024-03-21T06:38:58.483335+0000","last_became_active":"2024-03-21T06:25:58.494870+0000","last_became_peered":"2024-03-21T06:25:58.494870+0000","last_unstale":"2024-03-21T06:38:58.483335+0000","last_undegraded":"2024-03-21T06:38:58.483335+0000","last_fullsized":"2024-03-21T06:38:58.483335+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.530393+0000","last_change":"2024-03-21T06:25:58.500230+0000","last_active":"2024-03-21T06:38:58.530393+0000","last_peered":"2024-03-21T06:38:58.530393+0000","last_clean":"2024-03-21T06:38:58.530393+0000","last_became_active":"2024-03-21T06:25:58.488027+0000","last_became_peered":"2024-03-21T06:25:58.488027+0000","last_unstale":"2024-03-21T06:38:58.530393+0000","last_undegraded":"2024-03-21T06:38:58.530393+0000","last_fullsized":"2024-03-21T06:38:58.530393+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.488223+0000","last_change":"2024-03-21T06:25:58.505580+0000","last_active":"2024-03-21T06:38:58.488223+0000","last_peered":"2024-03-21T06:38:58.488223+0000","last_clean":"2024-03-21T06:38:58.488223+0000","last_became_active":"2024-03-21T06:25:58.498450+0000","last_became_peered":"2024-03-21T06:25:58.498450+0000","last_unstale":"2024-03-21T06:38:58.488223+0000","last_undegraded":"2024-03-21T06:38:58.488223+0000","last_fullsized":"2024-03-21T06:38:58.488223+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.720009+0000","last_change":"2024-03-21T06:25:58.515242+0000","last_active":"2024-03-21T06:38:29.720009+0000","last_peered":"2024-03-21T06:38:29.720009+0000","last_clean":"2024-03-21T06:38:29.720009+0000","last_became_active":"2024-03-21T06:25:58.503376+0000","last_became_peered":"2024-03-21T06:25:58.503376+0000","last_unstale":"2024-03-21T06:38:29.720009+0000","last_undegraded":"2024-03-21T06:38:29.720009+0000","last_fullsized":"2024-03-21T06:38:29.720009+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.517006+0000","last_change":"2024-03-21T06:25:58.500753+0000","last_active":"2024-03-21T06:38:27.517006+0000","last_peered":"2024-03-21T06:38:27.517006+0000","last_clean":"2024-03-21T06:38:27.517006+0000","last_became_active":"2024-03-21T06:25:58.497498+0000","last_became_peered":"2024-03-21T06:25:58.497498+0000","last_unstale":"2024-03-21T06:38:27.517006+0000","last_undegraded":"2024-03-21T06:38:27.517006+0000","last_fullsized":"2024-03-21T06:38:27.517006+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.695561+0000","last_change":"2024-03-21T06:25:58.509601+0000","last_active":"2024-03-21T06:38:29.695561+0000","last_peered":"2024-03-21T06:38:29.695561+0000","last_clean":"2024-03-21T06:38:29.695561+0000","last_became_active":"2024-03-21T06:25:58.498895+0000","last_became_peered":"2024-03-21T06:25:58.498895+0000","last_unstale":"2024-03-21T06:38:29.695561+0000","last_undegraded":"2024-03-21T06:38:29.695561+0000","last_fullsized":"2024-03-21T06:38:29.695561+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.505272+0000","last_change":"2024-03-21T06:25:58.496259+0000","last_active":"2024-03-21T06:38:58.505272+0000","last_peered":"2024-03-21T06:38:58.505272+0000","last_clean":"2024-03-21T06:38:58.505272+0000","last_became_active":"2024-03-21T06:25:58.486238+0000","last_became_peered":"2024-03-21T06:25:58.486238+0000","last_unstale":"2024-03-21T06:38:58.505272+0000","last_undegraded":"2024-03-21T06:38:58.505272+0000","last_fullsized":"2024-03-21T06:38:58.505272+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.711452+0000","last_change":"2024-03-21T06:25:58.524275+0000","last_active":"2024-03-21T06:38:29.711452+0000","last_peered":"2024-03-21T06:38:29.711452+0000","last_clean":"2024-03-21T06:38:29.711452+0000","last_became_active":"2024-03-21T06:25:58.513806+0000","last_became_peered":"2024-03-21T06:25:58.513806+0000","last_unstale":"2024-03-21T06:38:29.711452+0000","last_undegraded":"2024-03-21T06:38:29.711452+0000","last_fullsized":"2024-03-21T06:38:29.711452+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.707100+0000","last_change":"2024-03-21T06:25:58.540399+0000","last_active":"2024-03-21T06:38:29.707100+0000","last_peered":"2024-03-21T06:38:29.707100+0000","last_clean":"2024-03-21T06:38:29.707100+0000","last_became_active":"2024-03-21T06:25:58.537009+0000","last_became_peered":"2024-03-21T06:25:58.537009+0000","last_unstale":"2024-03-21T06:38:29.707100+0000","last_undegraded":"2024-03-21T06:38:29.707100+0000","last_fullsized":"2024-03-21T06:38:29.707100+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.482596+0000","last_change":"2024-03-21T06:25:58.514471+0000","last_active":"2024-03-21T06:38:58.482596+0000","last_peered":"2024-03-21T06:38:58.482596+0000","last_clean":"2024-03-21T06:38:58.482596+0000","last_became_active":"2024-03-21T06:25:58.510651+0000","last_became_peered":"2024-03-21T06:25:58.510651+0000","last_unstale":"2024-03-21T06:38:58.482596+0000","last_undegraded":"2024-03-21T06:38:58.482596+0000","last_fullsized":"2024-03-21T06:38:58.482596+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.529785+0000","last_change":"2024-03-21T06:25:58.551761+0000","last_active":"2024-03-21T06:38:58.529785+0000","last_peered":"2024-03-21T06:38:58.529785+0000","last_clean":"2024-03-21T06:38:58.529785+0000","last_became_active":"2024-03-21T06:25:58.548956+0000","last_became_peered":"2024-03-21T06:25:58.548956+0000","last_unstale":"2024-03-21T06:38:58.529785+0000","last_undegraded":"2024-03-21T06:38:58.529785+0000","last_fullsized":"2024-03-21T06:38:58.529785+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.499863+0000","last_change":"2024-03-21T06:25:58.490344+0000","last_active":"2024-03-21T06:38:58.499863+0000","last_peered":"2024-03-21T06:38:58.499863+0000","last_clean":"2024-03-21T06:38:58.499863+0000","last_became_active":"2024-03-21T06:25:58.482358+0000","last_became_peered":"2024-03-21T06:25:58.482358+0000","last_unstale":"2024-03-21T06:38:58.499863+0000","last_undegraded":"2024-03-21T06:38:58.499863+0000","last_fullsized":"2024-03-21T06:38:58.499863+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.718502+0000","last_change":"2024-03-21T06:25:58.527681+0000","last_active":"2024-03-21T06:38:29.718502+0000","last_peered":"2024-03-21T06:38:29.718502+0000","last_clean":"2024-03-21T06:38:29.718502+0000","last_became_active":"2024-03-21T06:25:58.521822+0000","last_became_peered":"2024-03-21T06:25:58.521822+0000","last_unstale":"2024-03-21T06:38:29.718502+0000","last_undegraded":"2024-03-21T06:38:29.718502+0000","last_fullsized":"2024-03-21T06:38:29.718502+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.526661+0000","last_change":"2024-03-21T06:25:58.490537+0000","last_active":"2024-03-21T06:38:27.526661+0000","last_peered":"2024-03-21T06:38:27.526661+0000","last_clean":"2024-03-21T06:38:27.526661+0000","last_became_active":"2024-03-21T06:25:58.484498+0000","last_became_peered":"2024-03-21T06:25:58.484498+0000","last_unstale":"2024-03-21T06:38:27.526661+0000","last_undegraded":"2024-03-21T06:38:27.526661+0000","last_fullsized":"2024-03-21T06:38:27.526661+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.734123+0000","last_change":"2024-03-21T06:25:58.482707+0000","last_active":"2024-03-21T06:38:29.734123+0000","last_peered":"2024-03-21T06:38:29.734123+0000","last_clean":"2024-03-21T06:38:29.734123+0000","last_became_active":"2024-03-21T06:25:58.474376+0000","last_became_peered":"2024-03-21T06:25:58.474376+0000","last_unstale":"2024-03-21T06:38:29.734123+0000","last_undegraded":"2024-03-21T06:38:29.734123+0000","last_fullsized":"2024-03-21T06:38:29.734123+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.754301+0000","last_change":"2024-03-21T06:25:58.483801+0000","last_active":"2024-03-21T06:38:29.754301+0000","last_peered":"2024-03-21T06:38:29.754301+0000","last_clean":"2024-03-21T06:38:29.754301+0000","last_became_active":"2024-03-21T06:25:58.477107+0000","last_became_peered":"2024-03-21T06:25:58.477107+0000","last_unstale":"2024-03-21T06:38:29.754301+0000","last_undegraded":"2024-03-21T06:38:29.754301+0000","last_fullsized":"2024-03-21T06:38:29.754301+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.736832+0000","last_change":"2024-03-21T06:25:58.494316+0000","last_active":"2024-03-21T06:38:29.736832+0000","last_peered":"2024-03-21T06:38:29.736832+0000","last_clean":"2024-03-21T06:38:29.736832+0000","last_became_active":"2024-03-21T06:25:58.490894+0000","last_became_peered":"2024-03-21T06:25:58.490894+0000","last_unstale":"2024-03-21T06:38:29.736832+0000","last_undegraded":"2024-03-21T06:38:29.736832+0000","last_fullsized":"2024-03-21T06:38:29.736832+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.523400+0000","last_change":"2024-03-21T06:25:58.508102+0000","last_active":"2024-03-21T06:38:27.523400+0000","last_peered":"2024-03-21T06:38:27.523400+0000","last_clean":"2024-03-21T06:38:27.523400+0000","last_became_active":"2024-03-21T06:25:58.505636+0000","last_became_peered":"2024-03-21T06:25:58.505636+0000","last_unstale":"2024-03-21T06:38:27.523400+0000","last_undegraded":"2024-03-21T06:38:27.523400+0000","last_fullsized":"2024-03-21T06:38:27.523400+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.714562+0000","last_change":"2024-03-21T06:25:58.512176+0000","last_active":"2024-03-21T06:38:29.714562+0000","last_peered":"2024-03-21T06:38:29.714562+0000","last_clean":"2024-03-21T06:38:29.714562+0000","last_became_active":"2024-03-21T06:25:58.501608+0000","last_became_peered":"2024-03-21T06:25:58.501608+0000","last_unstale":"2024-03-21T06:38:29.714562+0000","last_undegraded":"2024-03-21T06:38:29.714562+0000","last_fullsized":"2024-03-21T06:38:29.714562+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.715197+0000","last_change":"2024-03-21T06:25:58.517908+0000","last_active":"2024-03-21T06:38:29.715197+0000","last_peered":"2024-03-21T06:38:29.715197+0000","last_clean":"2024-03-21T06:38:29.715197+0000","last_became_active":"2024-03-21T06:25:58.508192+0000","last_became_peered":"2024-03-21T06:25:58.508192+0000","last_unstale":"2024-03-21T06:38:29.715197+0000","last_undegraded":"2024-03-21T06:38:29.715197+0000","last_fullsized":"2024-03-21T06:38:29.715197+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.517104+0000","last_change":"2024-03-21T06:25:58.501302+0000","last_active":"2024-03-21T06:38:58.517104+0000","last_peered":"2024-03-21T06:38:58.517104+0000","last_clean":"2024-03-21T06:38:58.517104+0000","last_became_active":"2024-03-21T06:25:58.491525+0000","last_became_peered":"2024-03-21T06:25:58.491525+0000","last_unstale":"2024-03-21T06:38:58.517104+0000","last_undegraded":"2024-03-21T06:38:58.517104+0000","last_fullsized":"2024-03-21T06:38:58.517104+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.700998+0000","last_change":"2024-03-21T06:25:58.516179+0000","last_active":"2024-03-21T06:38:29.700998+0000","last_peered":"2024-03-21T06:38:29.700998+0000","last_clean":"2024-03-21T06:38:29.700998+0000","last_became_active":"2024-03-21T06:25:58.506657+0000","last_became_peered":"2024-03-21T06:25:58.506657+0000","last_unstale":"2024-03-21T06:38:29.700998+0000","last_undegraded":"2024-03-21T06:38:29.700998+0000","last_fullsized":"2024-03-21T06:38:29.700998+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.490947+0000","last_change":"2024-03-21T06:25:58.521034+0000","last_active":"2024-03-21T06:38:58.490947+0000","last_peered":"2024-03-21T06:38:58.490947+0000","last_clean":"2024-03-21T06:38:58.490947+0000","last_became_active":"2024-03-21T06:25:58.517030+0000","last_became_peered":"2024-03-21T06:25:58.517030+0000","last_unstale":"2024-03-21T06:38:58.490947+0000","last_undegraded":"2024-03-21T06:38:58.490947+0000","last_fullsized":"2024-03-21T06:38:58.490947+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.493691+0000","last_change":"2024-03-21T06:25:58.531640+0000","last_active":"2024-03-21T06:38:58.493691+0000","last_peered":"2024-03-21T06:38:58.493691+0000","last_clean":"2024-03-21T06:38:58.493691+0000","last_became_active":"2024-03-21T06:25:58.528601+0000","last_became_peered":"2024-03-21T06:25:58.528601+0000","last_unstale":"2024-03-21T06:38:58.493691+0000","last_undegraded":"2024-03-21T06:38:58.493691+0000","last_fullsized":"2024-03-21T06:38:58.493691+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.532701+0000","last_change":"2024-03-21T06:25:58.509341+0000","last_active":"2024-03-21T06:38:58.532701+0000","last_peered":"2024-03-21T06:38:58.532701+0000","last_clean":"2024-03-21T06:38:58.532701+0000","last_became_active":"2024-03-21T06:25:58.503544+0000","last_became_peered":"2024-03-21T06:25:58.503544+0000","last_unstale":"2024-03-21T06:38:58.532701+0000","last_undegraded":"2024-03-21T06:38:58.532701+0000","last_fullsized":"2024-03-21T06:38:58.532701+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"236'7","reported_seq":1284,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.757963+0000","last_change":"2024-03-21T06:25:58.509187+0000","last_active":"2024-03-21T06:38:29.757963+0000","last_peered":"2024-03-21T06:38:29.757963+0000","last_clean":"2024-03-21T06:38:29.757963+0000","last_became_active":"2024-03-21T06:25:58.505585+0000","last_became_peered":"2024-03-21T06:25:58.505585+0000","last_unstale":"2024-03-21T06:38:29.757963+0000","last_undegraded":"2024-03-21T06:38:29.757963+0000","last_fullsized":"2024-03-21T06:38:29.757963+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.513968+0000","last_change":"2024-03-21T06:25:58.486325+0000","last_active":"2024-03-21T06:38:27.513968+0000","last_peered":"2024-03-21T06:38:27.513968+0000","last_clean":"2024-03-21T06:38:27.513968+0000","last_became_active":"2024-03-21T06:25:58.481996+0000","last_became_peered":"2024-03-21T06:25:58.481996+0000","last_unstale":"2024-03-21T06:38:27.513968+0000","last_undegraded":"2024-03-21T06:38:27.513968+0000","last_fullsized":"2024-03-21T06:38:27.513968+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1870,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.736177+0000","last_change":"2024-03-21T06:20:23.724343+0000","last_active":"2024-03-21T06:38:29.736177+0000","last_peered":"2024-03-21T06:38:29.736177+0000","last_clean":"2024-03-21T06:38:29.736177+0000","last_became_active":"2024-03-21T06:20:23.721083+0000","last_became_peered":"2024-03-21T06:20:23.721083+0000","last_unstale":"2024-03-21T06:38:29.736177+0000","last_undegraded":"2024-03-21T06:38:29.736177+0000","last_fullsized":"2024-03-21T06:38:29.736177+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1880,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.608992+0000","last_change":"2024-03-21T06:20:23.720094+0000","last_active":"2024-03-21T06:38:27.608992+0000","last_peered":"2024-03-21T06:38:27.608992+0000","last_clean":"2024-03-21T06:38:27.608992+0000","last_became_active":"2024-03-21T06:20:23.717296+0000","last_became_peered":"2024-03-21T06:20:23.717296+0000","last_unstale":"2024-03-21T06:38:27.608992+0000","last_undegraded":"2024-03-21T06:38:27.608992+0000","last_fullsized":"2024-03-21T06:38:27.608992+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1864,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.785694+0000","last_change":"2024-03-21T06:20:23.724647+0000","last_active":"2024-03-21T06:38:29.785694+0000","last_peered":"2024-03-21T06:38:29.785694+0000","last_clean":"2024-03-21T06:38:29.785694+0000","last_became_active":"2024-03-21T06:20:23.721198+0000","last_became_peered":"2024-03-21T06:20:23.721198+0000","last_unstale":"2024-03-21T06:38:29.785694+0000","last_undegraded":"2024-03-21T06:38:29.785694+0000","last_fullsized":"2024-03-21T06:38:29.785694+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1880,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.522321+0000","last_change":"2024-03-21T06:20:23.716953+0000","last_active":"2024-03-21T06:38:27.522321+0000","last_peered":"2024-03-21T06:38:27.522321+0000","last_clean":"2024-03-21T06:38:27.522321+0000","last_became_active":"2024-03-21T06:20:23.714274+0000","last_became_peered":"2024-03-21T06:20:23.714274+0000","last_unstale":"2024-03-21T06:38:27.522321+0000","last_undegraded":"2024-03-21T06:38:27.522321+0000","last_fullsized":"2024-03-21T06:38:27.522321+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1867,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.546801+0000","last_change":"2024-03-21T06:20:23.717551+0000","last_active":"2024-03-21T06:38:58.546801+0000","last_peered":"2024-03-21T06:38:58.546801+0000","last_clean":"2024-03-21T06:38:58.546801+0000","last_became_active":"2024-03-21T06:20:23.714766+0000","last_became_peered":"2024-03-21T06:20:23.714766+0000","last_unstale":"2024-03-21T06:38:58.546801+0000","last_undegraded":"2024-03-21T06:38:58.546801+0000","last_fullsized":"2024-03-21T06:38:58.546801+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1870,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.759598+0000","last_change":"2024-03-21T06:20:23.716172+0000","last_active":"2024-03-21T06:38:29.759598+0000","last_peered":"2024-03-21T06:38:29.759598+0000","last_clean":"2024-03-21T06:38:29.759598+0000","last_became_active":"2024-03-21T06:20:23.712781+0000","last_became_peered":"2024-03-21T06:20:23.712781+0000","last_unstale":"2024-03-21T06:38:29.759598+0000","last_undegraded":"2024-03-21T06:38:29.759598+0000","last_fullsized":"2024-03-21T06:38:29.759598+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1867,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.557390+0000","last_change":"2024-03-21T06:20:23.724311+0000","last_active":"2024-03-21T06:38:58.557390+0000","last_peered":"2024-03-21T06:38:58.557390+0000","last_clean":"2024-03-21T06:38:58.557390+0000","last_became_active":"2024-03-21T06:20:23.720894+0000","last_became_peered":"2024-03-21T06:20:23.720894+0000","last_unstale":"2024-03-21T06:38:58.557390+0000","last_undegraded":"2024-03-21T06:38:58.557390+0000","last_fullsized":"2024-03-21T06:38:58.557390+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1870,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.809884+0000","last_change":"2024-03-21T06:20:23.720175+0000","last_active":"2024-03-21T06:38:29.809884+0000","last_peered":"2024-03-21T06:38:29.809884+0000","last_clean":"2024-03-21T06:38:29.809884+0000","last_became_active":"2024-03-21T06:20:23.716768+0000","last_became_peered":"2024-03-21T06:20:23.716768+0000","last_unstale":"2024-03-21T06:38:29.809884+0000","last_undegraded":"2024-03-21T06:38:29.809884+0000","last_fullsized":"2024-03-21T06:38:29.809884+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":1997,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.703106+0000","last_change":"2024-03-21T06:20:21.697506+0000","last_active":"2024-03-21T06:38:29.703106+0000","last_peered":"2024-03-21T06:38:29.703106+0000","last_clean":"2024-03-21T06:38:29.703106+0000","last_became_active":"2024-03-21T06:20:21.690580+0000","last_became_peered":"2024-03-21T06:20:21.690580+0000","last_unstale":"2024-03-21T06:38:29.703106+0000","last_undegraded":"2024-03-21T06:38:29.703106+0000","last_fullsized":"2024-03-21T06:38:29.703106+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:20.658895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:20.658895+0000","last_clean_scrub_stamp":"2024-03-21T06:20:20.658895+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":71252,"kb_used_data":4044,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":104786348,"statfs":{"total":107374182400,"available":107301220352,"internally_reserved":0,"allocated":4141056,"data_stored":2006883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":70756,"kb_used_data":3548,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":104786844,"statfs":{"total":107374182400,"available":107301728256,"internally_reserved":0,"allocated":3633152,"data_stored":1449822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":70740,"kb_used_data":3532,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":104786860,"statfs":{"total":107374182400,"available":107301744640,"internally_reserved":0,"allocated":3616768,"data_stored":1448988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":75484,"kb_used_data":4116,"kb_used_omap":0,"kb_used_meta":71360,"kb_avail":104782116,"statfs":{"total":107374182400,"available":107296886784,"internally_reserved":0,"allocated":4214784,"data_stored":2051132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":73072640},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-21T06:39:00.491 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T06:39:00.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.594+0000 7f297bd5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:00.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.598+0000 7f297bd5c640 1 Processor -- start 2024-03-21T06:39:00.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.598+0000 7f297bd5c640 1 -- start start 2024-03-21T06:39:00.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.598+0000 7f297bd5c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f297409b5d0 0x7f2974130460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:00.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.598+0000 7f297bd5c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2974099dc0 con 0x7f29741309a0 2024-03-21T06:39:00.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.598+0000 7f297bd5c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f297409bae0 con 0x7f297409b5d0 2024-03-21T06:39:00.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.599+0000 7f2979af8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f297409b5d0 0x7f2974130460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:00.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.599+0000 7f29792f7640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f29741309a0 0x7f2974098190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:53104/0 (socket says 172.21.15.96:53104) 2024-03-21T06:39:00.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.599+0000 7f29792f7640 1 -- 172.21.15.96:0/690260966 learned_addr learned my addr 172.21.15.96:0/690260966 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:39:00.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.599+0000 7f2979af8640 1 -- 172.21.15.96:0/690260966 >> v1:172.21.15.96:6789/0 conn(0x7f29741309a0 legacy=0x7f2974098190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:00.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.599+0000 7f2979af8640 1 -- 172.21.15.96:0/690260966 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297409c0f0 con 0x7f297409b5d0 2024-03-21T06:39:00.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.599+0000 7f2979af8640 1 --2- 172.21.15.96:0/690260966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297409b5d0 0x7f2974130460 crc :-1 s=READY pgs=12983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9ba8a2d84c9240 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:00.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.599+0000 7f2978af6640 1 -- 172.21.15.96:0/690260966 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f296800ac70 con 0x7f297409b5d0 2024-03-21T06:39:00.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.599+0000 7f2978af6640 1 -- 172.21.15.96:0/690260966 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f296800add0 con 0x7f297409b5d0 2024-03-21T06:39:00.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.599+0000 7f2978af6640 1 -- 172.21.15.96:0/690260966 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f296800f470 con 0x7f297409b5d0 2024-03-21T06:39:00.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.600+0000 7f297bd5c640 1 -- 172.21.15.96:0/690260966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297409b5d0 msgr2=0x7f2974130460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:00.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.600+0000 7f297bd5c640 1 --2- 172.21.15.96:0/690260966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297409b5d0 0x7f2974130460 crc :-1 s=READY pgs=12983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:00.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.600+0000 7f297bd5c640 1 -- 172.21.15.96:0/690260966 shutdown_connections 2024-03-21T06:39:00.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.600+0000 7f297bd5c640 1 --2- 172.21.15.96:0/690260966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297409b5d0 0x7f2974130460 unknown :-1 s=CLOSED pgs=12983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:00.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.600+0000 7f297bd5c640 1 -- 172.21.15.96:0/690260966 >> 172.21.15.96:0/690260966 conn(0x7f297409fdb0 msgr2=0x7f29740a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:00.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.600+0000 7f297bd5c640 1 -- 172.21.15.96:0/690260966 shutdown_connections 2024-03-21T06:39:00.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.600+0000 7f297bd5c640 1 -- 172.21.15.96:0/690260966 wait complete. 2024-03-21T06:39:00.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.600+0000 7f297bd5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:00.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.601+0000 7f297bd5c640 1 Processor -- start 2024-03-21T06:39:00.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.601+0000 7f297bd5c640 1 -- start start 2024-03-21T06:39:00.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.602+0000 7f297bd5c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297409b5d0 0x7f297412d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:00.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.602+0000 7f297bd5c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2974073920 con 0x7f297409b5d0 2024-03-21T06:39:00.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.602+0000 7f2979af8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297409b5d0 0x7f297412d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:00.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.602+0000 7f2979af8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297409b5d0 0x7f297412d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55886/0 (socket says 172.21.15.96:55886) 2024-03-21T06:39:00.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.602+0000 7f2979af8640 1 -- 172.21.15.96:0/2318107548 learned_addr learned my addr 172.21.15.96:0/2318107548 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:39:00.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.602+0000 7f2979af8640 1 -- 172.21.15.96:0/2318107548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297412aed0 con 0x7f297409b5d0 2024-03-21T06:39:00.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.602+0000 7f2979af8640 1 --2- 172.21.15.96:0/2318107548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297409b5d0 0x7f297412d210 crc :-1 s=READY pgs=12984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:00.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.602+0000 7f29667fc640 1 -- 172.21.15.96:0/2318107548 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2968018040 con 0x7f297409b5d0 2024-03-21T06:39:00.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.602+0000 7f29667fc640 1 -- 172.21.15.96:0/2318107548 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f296800fdc0 con 0x7f297409b5d0 2024-03-21T06:39:00.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.602+0000 7f297bd5c640 1 -- 172.21.15.96:0/2318107548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297412b8e0 con 0x7f297409b5d0 2024-03-21T06:39:00.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.602+0000 7f29667fc640 1 -- 172.21.15.96:0/2318107548 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f2968011550 con 0x7f297409b5d0 2024-03-21T06:39:00.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.602+0000 7f297bd5c640 1 -- 172.21.15.96:0/2318107548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297412abb0 con 0x7f297409b5d0 2024-03-21T06:39:00.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.603+0000 7f29667fc640 1 -- 172.21.15.96:0/2318107548 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f2968011700 con 0x7f297409b5d0 2024-03-21T06:39:00.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.603+0000 7f297bd5c640 1 -- 172.21.15.96:0/2318107548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2974062f10 con 0x7f297409b5d0 2024-03-21T06:39:00.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.603+0000 7f29667fc640 1 --2- 172.21.15.96:0/2318107548 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f2958038260 0x7f295803a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:00.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.603+0000 7f29667fc640 1 -- 172.21.15.96:0/2318107548 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(713..713 src has 402..713) v4 ==== 3403+0+0 (crc 0 0 0) 0x7f2968049b30 con 0x7f297409b5d0 2024-03-21T06:39:00.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.604+0000 7f29792f7640 1 --2- 172.21.15.96:0/2318107548 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f2958038260 0x7f295803a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:00.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.604+0000 7f29792f7640 1 --2- 172.21.15.96:0/2318107548 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f2958038260 0x7f295803a770 crc :-1 s=READY pgs=3863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.608+0000 7f29667fc640 1 -- 172.21.15.96:0/2318107548 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f296800c250 con 0x7f297409b5d0 2024-03-21T06:39:00.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.778+0000 7f297bd5c640 1 -- 172.21.15.96:0/2318107548 --> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f297412ba70 con 0x7f2958038260 2024-03-21T06:39:00.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.782+0000 7f29667fc640 1 -- 172.21.15.96:0/2318107548 <== mgr.4099 v2:172.21.15.184:6800/2376803682 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+107144 (crc 0 0 0) 0x7f297412ba70 con 0x7f2958038260 2024-03-21T06:39:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:39:00.782 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-03-21T06:39:00.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.783+0000 7f297bd5c640 1 -- 172.21.15.96:0/2318107548 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f2958038260 msgr2=0x7f295803a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:00.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.783+0000 7f297bd5c640 1 --2- 172.21.15.96:0/2318107548 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f2958038260 0x7f295803a770 crc :-1 s=READY pgs=3863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:00.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.783+0000 7f297bd5c640 1 -- 172.21.15.96:0/2318107548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297409b5d0 msgr2=0x7f297412d210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:00.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.783+0000 7f297bd5c640 1 --2- 172.21.15.96:0/2318107548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297409b5d0 0x7f297412d210 crc :-1 s=READY pgs=12984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:00.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.783+0000 7f297bd5c640 1 -- 172.21.15.96:0/2318107548 shutdown_connections 2024-03-21T06:39:00.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.783+0000 7f297bd5c640 1 --2- 172.21.15.96:0/2318107548 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f2958038260 0x7f295803a770 unknown :-1 s=CLOSED pgs=3863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:00.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.783+0000 7f297bd5c640 1 --2- 172.21.15.96:0/2318107548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f297409b5d0 0x7f297412d210 unknown :-1 s=CLOSED pgs=12984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:00.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.783+0000 7f297bd5c640 1 -- 172.21.15.96:0/2318107548 >> 172.21.15.96:0/2318107548 conn(0x7f297409fdb0 msgr2=0x7f297412fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:00.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.783+0000 7f297bd5c640 1 -- 172.21.15.96:0/2318107548 shutdown_connections 2024-03-21T06:39:00.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.783+0000 7f297bd5c640 1 -- 172.21.15.96:0/2318107548 wait complete. 2024-03-21T06:39:00.793 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":1266,"stamp":"2024-03-21T06:38:59.478725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":288232,"kb_used_data":15240,"kb_used_omap":0,"kb_used_meta":272960,"kb_avail":419142168,"statfs":{"total":429496729600,"available":429201580032,"internally_reserved":0,"allocated":15605760,"data_stored":6956825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279511040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-295300,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-25,"num_write_kb":-300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.621107"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.760844+0000","last_change":"2024-03-21T06:25:58.485657+0000","last_active":"2024-03-21T06:38:29.760844+0000","last_peered":"2024-03-21T06:38:29.760844+0000","last_clean":"2024-03-21T06:38:29.760844+0000","last_became_active":"2024-03-21T06:25:58.480248+0000","last_became_peered":"2024-03-21T06:25:58.480248+0000","last_unstale":"2024-03-21T06:38:29.760844+0000","last_undegraded":"2024-03-21T06:38:29.760844+0000","last_fullsized":"2024-03-21T06:38:29.760844+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.500365+0000","last_change":"2024-03-21T06:25:58.492610+0000","last_active":"2024-03-21T06:38:58.500365+0000","last_peered":"2024-03-21T06:38:58.500365+0000","last_clean":"2024-03-21T06:38:58.500365+0000","last_became_active":"2024-03-21T06:25:58.484415+0000","last_became_peered":"2024-03-21T06:25:58.484415+0000","last_unstale":"2024-03-21T06:38:58.500365+0000","last_undegraded":"2024-03-21T06:38:58.500365+0000","last_fullsized":"2024-03-21T06:38:58.500365+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.521005+0000","last_change":"2024-03-21T06:25:58.492051+0000","last_active":"2024-03-21T06:38:27.521005+0000","last_peered":"2024-03-21T06:38:27.521005+0000","last_clean":"2024-03-21T06:38:27.521005+0000","last_became_active":"2024-03-21T06:25:58.487475+0000","last_became_peered":"2024-03-21T06:25:58.487475+0000","last_unstale":"2024-03-21T06:38:27.521005+0000","last_undegraded":"2024-03-21T06:38:27.521005+0000","last_fullsized":"2024-03-21T06:38:27.521005+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.713728+0000","last_change":"2024-03-21T06:25:58.528878+0000","last_active":"2024-03-21T06:38:29.713728+0000","last_peered":"2024-03-21T06:38:29.713728+0000","last_clean":"2024-03-21T06:38:29.713728+0000","last_became_active":"2024-03-21T06:25:58.519468+0000","last_became_peered":"2024-03-21T06:25:58.519468+0000","last_unstale":"2024-03-21T06:38:29.713728+0000","last_undegraded":"2024-03-21T06:38:29.713728+0000","last_fullsized":"2024-03-21T06:38:29.713728+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.747889+0000","last_change":"2024-03-21T06:25:58.530972+0000","last_active":"2024-03-21T06:38:29.747889+0000","last_peered":"2024-03-21T06:38:29.747889+0000","last_clean":"2024-03-21T06:38:29.747889+0000","last_became_active":"2024-03-21T06:25:58.527143+0000","last_became_peered":"2024-03-21T06:25:58.527143+0000","last_unstale":"2024-03-21T06:38:29.747889+0000","last_undegraded":"2024-03-21T06:38:29.747889+0000","last_fullsized":"2024-03-21T06:38:29.747889+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.483335+0000","last_change":"2024-03-21T06:25:58.506594+0000","last_active":"2024-03-21T06:38:58.483335+0000","last_peered":"2024-03-21T06:38:58.483335+0000","last_clean":"2024-03-21T06:38:58.483335+0000","last_became_active":"2024-03-21T06:25:58.494870+0000","last_became_peered":"2024-03-21T06:25:58.494870+0000","last_unstale":"2024-03-21T06:38:58.483335+0000","last_undegraded":"2024-03-21T06:38:58.483335+0000","last_fullsized":"2024-03-21T06:38:58.483335+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.530393+0000","last_change":"2024-03-21T06:25:58.500230+0000","last_active":"2024-03-21T06:38:58.530393+0000","last_peered":"2024-03-21T06:38:58.530393+0000","last_clean":"2024-03-21T06:38:58.530393+0000","last_became_active":"2024-03-21T06:25:58.488027+0000","last_became_peered":"2024-03-21T06:25:58.488027+0000","last_unstale":"2024-03-21T06:38:58.530393+0000","last_undegraded":"2024-03-21T06:38:58.530393+0000","last_fullsized":"2024-03-21T06:38:58.530393+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.488223+0000","last_change":"2024-03-21T06:25:58.505580+0000","last_active":"2024-03-21T06:38:58.488223+0000","last_peered":"2024-03-21T06:38:58.488223+0000","last_clean":"2024-03-21T06:38:58.488223+0000","last_became_active":"2024-03-21T06:25:58.498450+0000","last_became_peered":"2024-03-21T06:25:58.498450+0000","last_unstale":"2024-03-21T06:38:58.488223+0000","last_undegraded":"2024-03-21T06:38:58.488223+0000","last_fullsized":"2024-03-21T06:38:58.488223+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.720009+0000","last_change":"2024-03-21T06:25:58.515242+0000","last_active":"2024-03-21T06:38:29.720009+0000","last_peered":"2024-03-21T06:38:29.720009+0000","last_clean":"2024-03-21T06:38:29.720009+0000","last_became_active":"2024-03-21T06:25:58.503376+0000","last_became_peered":"2024-03-21T06:25:58.503376+0000","last_unstale":"2024-03-21T06:38:29.720009+0000","last_undegraded":"2024-03-21T06:38:29.720009+0000","last_fullsized":"2024-03-21T06:38:29.720009+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.517006+0000","last_change":"2024-03-21T06:25:58.500753+0000","last_active":"2024-03-21T06:38:27.517006+0000","last_peered":"2024-03-21T06:38:27.517006+0000","last_clean":"2024-03-21T06:38:27.517006+0000","last_became_active":"2024-03-21T06:25:58.497498+0000","last_became_peered":"2024-03-21T06:25:58.497498+0000","last_unstale":"2024-03-21T06:38:27.517006+0000","last_undegraded":"2024-03-21T06:38:27.517006+0000","last_fullsized":"2024-03-21T06:38:27.517006+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.695561+0000","last_change":"2024-03-21T06:25:58.509601+0000","last_active":"2024-03-21T06:38:29.695561+0000","last_peered":"2024-03-21T06:38:29.695561+0000","last_clean":"2024-03-21T06:38:29.695561+0000","last_became_active":"2024-03-21T06:25:58.498895+0000","last_became_peered":"2024-03-21T06:25:58.498895+0000","last_unstale":"2024-03-21T06:38:29.695561+0000","last_undegraded":"2024-03-21T06:38:29.695561+0000","last_fullsized":"2024-03-21T06:38:29.695561+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.505272+0000","last_change":"2024-03-21T06:25:58.496259+0000","last_active":"2024-03-21T06:38:58.505272+0000","last_peered":"2024-03-21T06:38:58.505272+0000","last_clean":"2024-03-21T06:38:58.505272+0000","last_became_active":"2024-03-21T06:25:58.486238+0000","last_became_peered":"2024-03-21T06:25:58.486238+0000","last_unstale":"2024-03-21T06:38:58.505272+0000","last_undegraded":"2024-03-21T06:38:58.505272+0000","last_fullsized":"2024-03-21T06:38:58.505272+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.711452+0000","last_change":"2024-03-21T06:25:58.524275+0000","last_active":"2024-03-21T06:38:29.711452+0000","last_peered":"2024-03-21T06:38:29.711452+0000","last_clean":"2024-03-21T06:38:29.711452+0000","last_became_active":"2024-03-21T06:25:58.513806+0000","last_became_peered":"2024-03-21T06:25:58.513806+0000","last_unstale":"2024-03-21T06:38:29.711452+0000","last_undegraded":"2024-03-21T06:38:29.711452+0000","last_fullsized":"2024-03-21T06:38:29.711452+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.707100+0000","last_change":"2024-03-21T06:25:58.540399+0000","last_active":"2024-03-21T06:38:29.707100+0000","last_peered":"2024-03-21T06:38:29.707100+0000","last_clean":"2024-03-21T06:38:29.707100+0000","last_became_active":"2024-03-21T06:25:58.537009+0000","last_became_peered":"2024-03-21T06:25:58.537009+0000","last_unstale":"2024-03-21T06:38:29.707100+0000","last_undegraded":"2024-03-21T06:38:29.707100+0000","last_fullsized":"2024-03-21T06:38:29.707100+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.482596+0000","last_change":"2024-03-21T06:25:58.514471+0000","last_active":"2024-03-21T06:38:58.482596+0000","last_peered":"2024-03-21T06:38:58.482596+0000","last_clean":"2024-03-21T06:38:58.482596+0000","last_became_active":"2024-03-21T06:25:58.510651+0000","last_became_peered":"2024-03-21T06:25:58.510651+0000","last_unstale":"2024-03-21T06:38:58.482596+0000","last_undegraded":"2024-03-21T06:38:58.482596+0000","last_fullsized":"2024-03-21T06:38:58.482596+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.529785+0000","last_change":"2024-03-21T06:25:58.551761+0000","last_active":"2024-03-21T06:38:58.529785+0000","last_peered":"2024-03-21T06:38:58.529785+0000","last_clean":"2024-03-21T06:38:58.529785+0000","last_became_active":"2024-03-21T06:25:58.548956+0000","last_became_peered":"2024-03-21T06:25:58.548956+0000","last_unstale":"2024-03-21T06:38:58.529785+0000","last_undegraded":"2024-03-21T06:38:58.529785+0000","last_fullsized":"2024-03-21T06:38:58.529785+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.499863+0000","last_change":"2024-03-21T06:25:58.490344+0000","last_active":"2024-03-21T06:38:58.499863+0000","last_peered":"2024-03-21T06:38:58.499863+0000","last_clean":"2024-03-21T06:38:58.499863+0000","last_became_active":"2024-03-21T06:25:58.482358+0000","last_became_peered":"2024-03-21T06:25:58.482358+0000","last_unstale":"2024-03-21T06:38:58.499863+0000","last_undegraded":"2024-03-21T06:38:58.499863+0000","last_fullsized":"2024-03-21T06:38:58.499863+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.718502+0000","last_change":"2024-03-21T06:25:58.527681+0000","last_active":"2024-03-21T06:38:29.718502+0000","last_peered":"2024-03-21T06:38:29.718502+0000","last_clean":"2024-03-21T06:38:29.718502+0000","last_became_active":"2024-03-21T06:25:58.521822+0000","last_became_peered":"2024-03-21T06:25:58.521822+0000","last_unstale":"2024-03-21T06:38:29.718502+0000","last_undegraded":"2024-03-21T06:38:29.718502+0000","last_fullsized":"2024-03-21T06:38:29.718502+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.526661+0000","last_change":"2024-03-21T06:25:58.490537+0000","last_active":"2024-03-21T06:38:27.526661+0000","last_peered":"2024-03-21T06:38:27.526661+0000","last_clean":"2024-03-21T06:38:27.526661+0000","last_became_active":"2024-03-21T06:25:58.484498+0000","last_became_peered":"2024-03-21T06:25:58.484498+0000","last_unstale":"2024-03-21T06:38:27.526661+0000","last_undegraded":"2024-03-21T06:38:27.526661+0000","last_fullsized":"2024-03-21T06:38:27.526661+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.734123+0000","last_change":"2024-03-21T06:25:58.482707+0000","last_active":"2024-03-21T06:38:29.734123+0000","last_peered":"2024-03-21T06:38:29.734123+0000","last_clean":"2024-03-21T06:38:29.734123+0000","last_became_active":"2024-03-21T06:25:58.474376+0000","last_became_peered":"2024-03-21T06:25:58.474376+0000","last_unstale":"2024-03-21T06:38:29.734123+0000","last_undegraded":"2024-03-21T06:38:29.734123+0000","last_fullsized":"2024-03-21T06:38:29.734123+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.754301+0000","last_change":"2024-03-21T06:25:58.483801+0000","last_active":"2024-03-21T06:38:29.754301+0000","last_peered":"2024-03-21T06:38:29.754301+0000","last_clean":"2024-03-21T06:38:29.754301+0000","last_became_active":"2024-03-21T06:25:58.477107+0000","last_became_peered":"2024-03-21T06:25:58.477107+0000","last_unstale":"2024-03-21T06:38:29.754301+0000","last_undegraded":"2024-03-21T06:38:29.754301+0000","last_fullsized":"2024-03-21T06:38:29.754301+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.736832+0000","last_change":"2024-03-21T06:25:58.494316+0000","last_active":"2024-03-21T06:38:29.736832+0000","last_peered":"2024-03-21T06:38:29.736832+0000","last_clean":"2024-03-21T06:38:29.736832+0000","last_became_active":"2024-03-21T06:25:58.490894+0000","last_became_peered":"2024-03-21T06:25:58.490894+0000","last_unstale":"2024-03-21T06:38:29.736832+0000","last_undegraded":"2024-03-21T06:38:29.736832+0000","last_fullsized":"2024-03-21T06:38:29.736832+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.523400+0000","last_change":"2024-03-21T06:25:58.508102+0000","last_active":"2024-03-21T06:38:27.523400+0000","last_peered":"2024-03-21T06:38:27.523400+0000","last_clean":"2024-03-21T06:38:27.523400+0000","last_became_active":"2024-03-21T06:25:58.505636+0000","last_became_peered":"2024-03-21T06:25:58.505636+0000","last_unstale":"2024-03-21T06:38:27.523400+0000","last_undegraded":"2024-03-21T06:38:27.523400+0000","last_fullsized":"2024-03-21T06:38:27.523400+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.714562+0000","last_change":"2024-03-21T06:25:58.512176+0000","last_active":"2024-03-21T06:38:29.714562+0000","last_peered":"2024-03-21T06:38:29.714562+0000","last_clean":"2024-03-21T06:38:29.714562+0000","last_became_active":"2024-03-21T06:25:58.501608+0000","last_became_peered":"2024-03-21T06:25:58.501608+0000","last_unstale":"2024-03-21T06:38:29.714562+0000","last_undegraded":"2024-03-21T06:38:29.714562+0000","last_fullsized":"2024-03-21T06:38:29.714562+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.715197+0000","last_change":"2024-03-21T06:25:58.517908+0000","last_active":"2024-03-21T06:38:29.715197+0000","last_peered":"2024-03-21T06:38:29.715197+0000","last_clean":"2024-03-21T06:38:29.715197+0000","last_became_active":"2024-03-21T06:25:58.508192+0000","last_became_peered":"2024-03-21T06:25:58.508192+0000","last_unstale":"2024-03-21T06:38:29.715197+0000","last_undegraded":"2024-03-21T06:38:29.715197+0000","last_fullsized":"2024-03-21T06:38:29.715197+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.517104+0000","last_change":"2024-03-21T06:25:58.501302+0000","last_active":"2024-03-21T06:38:58.517104+0000","last_peered":"2024-03-21T06:38:58.517104+0000","last_clean":"2024-03-21T06:38:58.517104+0000","last_became_active":"2024-03-21T06:25:58.491525+0000","last_became_peered":"2024-03-21T06:25:58.491525+0000","last_unstale":"2024-03-21T06:38:58.517104+0000","last_undegraded":"2024-03-21T06:38:58.517104+0000","last_fullsized":"2024-03-21T06:38:58.517104+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.700998+0000","last_change":"2024-03-21T06:25:58.516179+0000","last_active":"2024-03-21T06:38:29.700998+0000","last_peered":"2024-03-21T06:38:29.700998+0000","last_clean":"2024-03-21T06:38:29.700998+0000","last_became_active":"2024-03-21T06:25:58.506657+0000","last_became_peered":"2024-03-21T06:25:58.506657+0000","last_unstale":"2024-03-21T06:38:29.700998+0000","last_undegraded":"2024-03-21T06:38:29.700998+0000","last_fullsized":"2024-03-21T06:38:29.700998+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.490947+0000","last_change":"2024-03-21T06:25:58.521034+0000","last_active":"2024-03-21T06:38:58.490947+0000","last_peered":"2024-03-21T06:38:58.490947+0000","last_clean":"2024-03-21T06:38:58.490947+0000","last_became_active":"2024-03-21T06:25:58.517030+0000","last_became_peered":"2024-03-21T06:25:58.517030+0000","last_unstale":"2024-03-21T06:38:58.490947+0000","last_undegraded":"2024-03-21T06:38:58.490947+0000","last_fullsized":"2024-03-21T06:38:58.490947+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.493691+0000","last_change":"2024-03-21T06:25:58.531640+0000","last_active":"2024-03-21T06:38:58.493691+0000","last_peered":"2024-03-21T06:38:58.493691+0000","last_clean":"2024-03-21T06:38:58.493691+0000","last_became_active":"2024-03-21T06:25:58.528601+0000","last_became_peered":"2024-03-21T06:25:58.528601+0000","last_unstale":"2024-03-21T06:38:58.493691+0000","last_undegraded":"2024-03-21T06:38:58.493691+0000","last_fullsized":"2024-03-21T06:38:58.493691+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.532701+0000","last_change":"2024-03-21T06:25:58.509341+0000","last_active":"2024-03-21T06:38:58.532701+0000","last_peered":"2024-03-21T06:38:58.532701+0000","last_clean":"2024-03-21T06:38:58.532701+0000","last_became_active":"2024-03-21T06:25:58.503544+0000","last_became_peered":"2024-03-21T06:25:58.503544+0000","last_unstale":"2024-03-21T06:38:58.532701+0000","last_undegraded":"2024-03-21T06:38:58.532701+0000","last_fullsized":"2024-03-21T06:38:58.532701+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"236'7","reported_seq":1284,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.757963+0000","last_change":"2024-03-21T06:25:58.509187+0000","last_active":"2024-03-21T06:38:29.757963+0000","last_peered":"2024-03-21T06:38:29.757963+0000","last_clean":"2024-03-21T06:38:29.757963+0000","last_became_active":"2024-03-21T06:25:58.505585+0000","last_became_peered":"2024-03-21T06:25:58.505585+0000","last_unstale":"2024-03-21T06:38:29.757963+0000","last_undegraded":"2024-03-21T06:38:29.757963+0000","last_fullsized":"2024-03-21T06:38:29.757963+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.513968+0000","last_change":"2024-03-21T06:25:58.486325+0000","last_active":"2024-03-21T06:38:27.513968+0000","last_peered":"2024-03-21T06:38:27.513968+0000","last_clean":"2024-03-21T06:38:27.513968+0000","last_became_active":"2024-03-21T06:25:58.481996+0000","last_became_peered":"2024-03-21T06:25:58.481996+0000","last_unstale":"2024-03-21T06:38:27.513968+0000","last_undegraded":"2024-03-21T06:38:27.513968+0000","last_fullsized":"2024-03-21T06:38:27.513968+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1870,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.736177+0000","last_change":"2024-03-21T06:20:23.724343+0000","last_active":"2024-03-21T06:38:29.736177+0000","last_peered":"2024-03-21T06:38:29.736177+0000","last_clean":"2024-03-21T06:38:29.736177+0000","last_became_active":"2024-03-21T06:20:23.721083+0000","last_became_peered":"2024-03-21T06:20:23.721083+0000","last_unstale":"2024-03-21T06:38:29.736177+0000","last_undegraded":"2024-03-21T06:38:29.736177+0000","last_fullsized":"2024-03-21T06:38:29.736177+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1880,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.608992+0000","last_change":"2024-03-21T06:20:23.720094+0000","last_active":"2024-03-21T06:38:27.608992+0000","last_peered":"2024-03-21T06:38:27.608992+0000","last_clean":"2024-03-21T06:38:27.608992+0000","last_became_active":"2024-03-21T06:20:23.717296+0000","last_became_peered":"2024-03-21T06:20:23.717296+0000","last_unstale":"2024-03-21T06:38:27.608992+0000","last_undegraded":"2024-03-21T06:38:27.608992+0000","last_fullsized":"2024-03-21T06:38:27.608992+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1864,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.785694+0000","last_change":"2024-03-21T06:20:23.724647+0000","last_active":"2024-03-21T06:38:29.785694+0000","last_peered":"2024-03-21T06:38:29.785694+0000","last_clean":"2024-03-21T06:38:29.785694+0000","last_became_active":"2024-03-21T06:20:23.721198+0000","last_became_peered":"2024-03-21T06:20:23.721198+0000","last_unstale":"2024-03-21T06:38:29.785694+0000","last_undegraded":"2024-03-21T06:38:29.785694+0000","last_fullsized":"2024-03-21T06:38:29.785694+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1880,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.522321+0000","last_change":"2024-03-21T06:20:23.716953+0000","last_active":"2024-03-21T06:38:27.522321+0000","last_peered":"2024-03-21T06:38:27.522321+0000","last_clean":"2024-03-21T06:38:27.522321+0000","last_became_active":"2024-03-21T06:20:23.714274+0000","last_became_peered":"2024-03-21T06:20:23.714274+0000","last_unstale":"2024-03-21T06:38:27.522321+0000","last_undegraded":"2024-03-21T06:38:27.522321+0000","last_fullsized":"2024-03-21T06:38:27.522321+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1867,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.546801+0000","last_change":"2024-03-21T06:20:23.717551+0000","last_active":"2024-03-21T06:38:58.546801+0000","last_peered":"2024-03-21T06:38:58.546801+0000","last_clean":"2024-03-21T06:38:58.546801+0000","last_became_active":"2024-03-21T06:20:23.714766+0000","last_became_peered":"2024-03-21T06:20:23.714766+0000","last_unstale":"2024-03-21T06:38:58.546801+0000","last_undegraded":"2024-03-21T06:38:58.546801+0000","last_fullsized":"2024-03-21T06:38:58.546801+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1870,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.759598+0000","last_change":"2024-03-21T06:20:23.716172+0000","last_active":"2024-03-21T06:38:29.759598+0000","last_peered":"2024-03-21T06:38:29.759598+0000","last_clean":"2024-03-21T06:38:29.759598+0000","last_became_active":"2024-03-21T06:20:23.712781+0000","last_became_peered":"2024-03-21T06:20:23.712781+0000","last_unstale":"2024-03-21T06:38:29.759598+0000","last_undegraded":"2024-03-21T06:38:29.759598+0000","last_fullsized":"2024-03-21T06:38:29.759598+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1867,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.557390+0000","last_change":"2024-03-21T06:20:23.724311+0000","last_active":"2024-03-21T06:38:58.557390+0000","last_peered":"2024-03-21T06:38:58.557390+0000","last_clean":"2024-03-21T06:38:58.557390+0000","last_became_active":"2024-03-21T06:20:23.720894+0000","last_became_peered":"2024-03-21T06:20:23.720894+0000","last_unstale":"2024-03-21T06:38:58.557390+0000","last_undegraded":"2024-03-21T06:38:58.557390+0000","last_fullsized":"2024-03-21T06:38:58.557390+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1870,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.809884+0000","last_change":"2024-03-21T06:20:23.720175+0000","last_active":"2024-03-21T06:38:29.809884+0000","last_peered":"2024-03-21T06:38:29.809884+0000","last_clean":"2024-03-21T06:38:29.809884+0000","last_became_active":"2024-03-21T06:20:23.716768+0000","last_became_peered":"2024-03-21T06:20:23.716768+0000","last_unstale":"2024-03-21T06:38:29.809884+0000","last_undegraded":"2024-03-21T06:38:29.809884+0000","last_fullsized":"2024-03-21T06:38:29.809884+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":1997,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.703106+0000","last_change":"2024-03-21T06:20:21.697506+0000","last_active":"2024-03-21T06:38:29.703106+0000","last_peered":"2024-03-21T06:38:29.703106+0000","last_clean":"2024-03-21T06:38:29.703106+0000","last_became_active":"2024-03-21T06:20:21.690580+0000","last_became_peered":"2024-03-21T06:20:21.690580+0000","last_unstale":"2024-03-21T06:38:29.703106+0000","last_undegraded":"2024-03-21T06:38:29.703106+0000","last_fullsized":"2024-03-21T06:38:29.703106+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:20.658895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:20.658895+0000","last_clean_scrub_stamp":"2024-03-21T06:20:20.658895+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":71252,"kb_used_data":4044,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":104786348,"statfs":{"total":107374182400,"available":107301220352,"internally_reserved":0,"allocated":4141056,"data_stored":2006883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":70756,"kb_used_data":3548,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":104786844,"statfs":{"total":107374182400,"available":107301728256,"internally_reserved":0,"allocated":3633152,"data_stored":1449822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":70740,"kb_used_data":3532,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":104786860,"statfs":{"total":107374182400,"available":107301744640,"internally_reserved":0,"allocated":3616768,"data_stored":1448988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":75484,"kb_used_data":4116,"kb_used_omap":0,"kb_used_meta":71360,"kb_avail":104782116,"statfs":{"total":107374182400,"available":107296886784,"internally_reserved":0,"allocated":4214784,"data_stored":2051132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":73072640},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-21T06:39:00.795 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-21T06:39:00.795 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T06:39:00.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.898+0000 7fd91d42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:00.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.902+0000 7fd91d42d640 1 Processor -- start 2024-03-21T06:39:00.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.902+0000 7fd91d42d640 1 -- start start 2024-03-21T06:39:00.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.903+0000 7fd91d42d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd91809b460 0x7fd918136460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:00.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.903+0000 7fd91d42d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd918099960 con 0x7fd91806d9e0 2024-03-21T06:39:00.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.903+0000 7fd91d42d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91809bf80 con 0x7fd91809b460 2024-03-21T06:39:00.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.903+0000 7fd90ffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd91809b460 0x7fd918136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:00.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.903+0000 7fd916ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fd91806d9e0 0x7fd91809ad50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:53120/0 (socket says 172.21.15.96:53120) 2024-03-21T06:39:00.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.903+0000 7fd916ffd640 1 -- 172.21.15.96:0/142643397 learned_addr learned my addr 172.21.15.96:0/142643397 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:39:00.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.903+0000 7fd90ffff640 1 -- 172.21.15.96:0/142643397 >> v1:172.21.15.96:6789/0 conn(0x7fd91806d9e0 legacy=0x7fd91809ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:00.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.903+0000 7fd9167fc640 1 -- 172.21.15.96:0/142643397 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105014856 0 0) 0x7fd918099960 con 0x7fd91806d9e0 2024-03-21T06:39:00.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.903+0000 7fd90ffff640 1 -- 172.21.15.96:0/142643397 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91809c1c0 con 0x7fd91809b460 2024-03-21T06:39:00.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.903+0000 7fd90ffff640 1 --2- 172.21.15.96:0/142643397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd91809b460 0x7fd918136460 crc :-1 s=READY pgs=12986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939d399c1fefde0d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:00.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.903+0000 7fd9167fc640 1 -- 172.21.15.96:0/142643397 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd900009ce0 con 0x7fd91809b460 2024-03-21T06:39:00.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.903+0000 7fd9167fc640 1 -- 172.21.15.96:0/142643397 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd900009e40 con 0x7fd91809b460 2024-03-21T06:39:00.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.903+0000 7fd9167fc640 1 -- 172.21.15.96:0/142643397 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd90000e360 con 0x7fd91809b460 2024-03-21T06:39:00.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.904+0000 7fd91d42d640 1 -- 172.21.15.96:0/142643397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd91809b460 msgr2=0x7fd918136460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:00.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.904+0000 7fd91d42d640 1 --2- 172.21.15.96:0/142643397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd91809b460 0x7fd918136460 crc :-1 s=READY pgs=12986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:00.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.904+0000 7fd91d42d640 1 -- 172.21.15.96:0/142643397 shutdown_connections 2024-03-21T06:39:00.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.904+0000 7fd91d42d640 1 --2- 172.21.15.96:0/142643397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd91809b460 0x7fd918136460 unknown :-1 s=CLOSED pgs=12986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:00.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.904+0000 7fd91d42d640 1 -- 172.21.15.96:0/142643397 >> 172.21.15.96:0/142643397 conn(0x7fd91809fdb0 msgr2=0x7fd9180a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:00.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.904+0000 7fd91d42d640 1 -- 172.21.15.96:0/142643397 shutdown_connections 2024-03-21T06:39:00.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.904+0000 7fd91d42d640 1 -- 172.21.15.96:0/142643397 wait complete. 2024-03-21T06:39:00.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.904+0000 7fd91d42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:00.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.905+0000 7fd91d42d640 1 Processor -- start 2024-03-21T06:39:00.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.905+0000 7fd91d42d640 1 -- start start 2024-03-21T06:39:00.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.906+0000 7fd91d42d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd91809b460 0x7fd9181c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:00.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.906+0000 7fd91d42d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91812d7f0 con 0x7fd91809b460 2024-03-21T06:39:00.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.906+0000 7fd916ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd91809b460 0x7fd9181c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:00.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.906+0000 7fd916ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd91809b460 0x7fd9181c53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55906/0 (socket says 172.21.15.96:55906) 2024-03-21T06:39:00.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.906+0000 7fd916ffd640 1 -- 172.21.15.96:0/1350344634 learned_addr learned my addr 172.21.15.96:0/1350344634 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:39:00.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.906+0000 7fd916ffd640 1 -- 172.21.15.96:0/1350344634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91812dab0 con 0x7fd91809b460 2024-03-21T06:39:00.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.906+0000 7fd916ffd640 1 --2- 172.21.15.96:0/1350344634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd91809b460 0x7fd9181c53a0 crc :-1 s=READY pgs=12987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:00.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.907+0000 7fd90f7fe640 1 -- 172.21.15.96:0/1350344634 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd908004af0 con 0x7fd91809b460 2024-03-21T06:39:00.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.907+0000 7fd90f7fe640 1 -- 172.21.15.96:0/1350344634 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd908004c50 con 0x7fd91809b460 2024-03-21T06:39:00.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.907+0000 7fd91d42d640 1 -- 172.21.15.96:0/1350344634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd918131c00 con 0x7fd91809b460 2024-03-21T06:39:00.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.907+0000 7fd90f7fe640 1 -- 172.21.15.96:0/1350344634 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd908007120 con 0x7fd91809b460 2024-03-21T06:39:00.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.907+0000 7fd91d42d640 1 -- 172.21.15.96:0/1350344634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd918130090 con 0x7fd91809b460 2024-03-21T06:39:00.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.907+0000 7fd90f7fe640 1 -- 172.21.15.96:0/1350344634 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fd9080072d0 con 0x7fd91809b460 2024-03-21T06:39:00.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.908+0000 7fd90f7fe640 1 --2- 172.21.15.96:0/1350344634 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd8f8038260 0x7fd8f803a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:00.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.908+0000 7fd90f7fe640 1 -- 172.21.15.96:0/1350344634 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(713..713 src has 402..713) v4 ==== 3403+0+0 (crc 0 0 0) 0x7fd908047cb0 con 0x7fd91809b460 2024-03-21T06:39:00.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.908+0000 7fd91d42d640 1 -- 172.21.15.96:0/1350344634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd918062f10 con 0x7fd91809b460 2024-03-21T06:39:00.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.908+0000 7fd90ffff640 1 --2- 172.21.15.96:0/1350344634 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd8f8038260 0x7fd8f803a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:00.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.908+0000 7fd90ffff640 1 --2- 172.21.15.96:0/1350344634 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd8f8038260 0x7fd8f803a770 crc :-1 s=READY pgs=3864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:00.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:00.913+0000 7fd90f7fe640 1 -- 172.21.15.96:0/1350344634 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fd90800f230 con 0x7fd91809b460 2024-03-21T06:39:01.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.083+0000 7fd91d42d640 1 -- 172.21.15.96:0/1350344634 --> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd918131a20 con 0x7fd8f8038260 2024-03-21T06:39:01.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.087+0000 7fd90f7fe640 1 -- 172.21.15.96:0/1350344634 <== mgr.4099 v2:172.21.15.184:6800/2376803682 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+107144 (crc 0 0 0) 0x7fd918131a20 con 0x7fd8f8038260 2024-03-21T06:39:01.087 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:39:01.087 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-03-21T06:39:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.089+0000 7fd91d42d640 1 -- 172.21.15.96:0/1350344634 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd8f8038260 msgr2=0x7fd8f803a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.089+0000 7fd91d42d640 1 --2- 172.21.15.96:0/1350344634 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd8f8038260 0x7fd8f803a770 crc :-1 s=READY pgs=3864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.089+0000 7fd91d42d640 1 -- 172.21.15.96:0/1350344634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd91809b460 msgr2=0x7fd9181c53a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.089+0000 7fd91d42d640 1 --2- 172.21.15.96:0/1350344634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd91809b460 0x7fd9181c53a0 crc :-1 s=READY pgs=12987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.089+0000 7fd91d42d640 1 -- 172.21.15.96:0/1350344634 shutdown_connections 2024-03-21T06:39:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.089+0000 7fd91d42d640 1 --2- 172.21.15.96:0/1350344634 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd8f8038260 0x7fd8f803a770 unknown :-1 s=CLOSED pgs=3864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.089+0000 7fd91d42d640 1 --2- 172.21.15.96:0/1350344634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd91809b460 0x7fd9181c53a0 unknown :-1 s=CLOSED pgs=12987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.089+0000 7fd91d42d640 1 -- 172.21.15.96:0/1350344634 >> 172.21.15.96:0/1350344634 conn(0x7fd91809fdb0 msgr2=0x7fd918126f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.089+0000 7fd91d42d640 1 -- 172.21.15.96:0/1350344634 shutdown_connections 2024-03-21T06:39:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.089+0000 7fd91d42d640 1 -- 172.21.15.96:0/1350344634 wait complete. 2024-03-21T06:39:01.099 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":1267,"stamp":"2024-03-21T06:39:00.860257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":288232,"kb_used_data":15240,"kb_used_omap":0,"kb_used_meta":272960,"kb_avail":419142168,"statfs":{"total":429496729600,"available":429201580032,"internally_reserved":0,"allocated":15605760,"data_stored":6956825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279511040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-295300,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-25,"num_write_kb":-300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002106"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.760844+0000","last_change":"2024-03-21T06:25:58.485657+0000","last_active":"2024-03-21T06:38:29.760844+0000","last_peered":"2024-03-21T06:38:29.760844+0000","last_clean":"2024-03-21T06:38:29.760844+0000","last_became_active":"2024-03-21T06:25:58.480248+0000","last_became_peered":"2024-03-21T06:25:58.480248+0000","last_unstale":"2024-03-21T06:38:29.760844+0000","last_undegraded":"2024-03-21T06:38:29.760844+0000","last_fullsized":"2024-03-21T06:38:29.760844+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.500365+0000","last_change":"2024-03-21T06:25:58.492610+0000","last_active":"2024-03-21T06:38:58.500365+0000","last_peered":"2024-03-21T06:38:58.500365+0000","last_clean":"2024-03-21T06:38:58.500365+0000","last_became_active":"2024-03-21T06:25:58.484415+0000","last_became_peered":"2024-03-21T06:25:58.484415+0000","last_unstale":"2024-03-21T06:38:58.500365+0000","last_undegraded":"2024-03-21T06:38:58.500365+0000","last_fullsized":"2024-03-21T06:38:58.500365+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.521005+0000","last_change":"2024-03-21T06:25:58.492051+0000","last_active":"2024-03-21T06:38:27.521005+0000","last_peered":"2024-03-21T06:38:27.521005+0000","last_clean":"2024-03-21T06:38:27.521005+0000","last_became_active":"2024-03-21T06:25:58.487475+0000","last_became_peered":"2024-03-21T06:25:58.487475+0000","last_unstale":"2024-03-21T06:38:27.521005+0000","last_undegraded":"2024-03-21T06:38:27.521005+0000","last_fullsized":"2024-03-21T06:38:27.521005+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.713728+0000","last_change":"2024-03-21T06:25:58.528878+0000","last_active":"2024-03-21T06:38:29.713728+0000","last_peered":"2024-03-21T06:38:29.713728+0000","last_clean":"2024-03-21T06:38:29.713728+0000","last_became_active":"2024-03-21T06:25:58.519468+0000","last_became_peered":"2024-03-21T06:25:58.519468+0000","last_unstale":"2024-03-21T06:38:29.713728+0000","last_undegraded":"2024-03-21T06:38:29.713728+0000","last_fullsized":"2024-03-21T06:38:29.713728+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.747889+0000","last_change":"2024-03-21T06:25:58.530972+0000","last_active":"2024-03-21T06:38:29.747889+0000","last_peered":"2024-03-21T06:38:29.747889+0000","last_clean":"2024-03-21T06:38:29.747889+0000","last_became_active":"2024-03-21T06:25:58.527143+0000","last_became_peered":"2024-03-21T06:25:58.527143+0000","last_unstale":"2024-03-21T06:38:29.747889+0000","last_undegraded":"2024-03-21T06:38:29.747889+0000","last_fullsized":"2024-03-21T06:38:29.747889+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.483335+0000","last_change":"2024-03-21T06:25:58.506594+0000","last_active":"2024-03-21T06:38:58.483335+0000","last_peered":"2024-03-21T06:38:58.483335+0000","last_clean":"2024-03-21T06:38:58.483335+0000","last_became_active":"2024-03-21T06:25:58.494870+0000","last_became_peered":"2024-03-21T06:25:58.494870+0000","last_unstale":"2024-03-21T06:38:58.483335+0000","last_undegraded":"2024-03-21T06:38:58.483335+0000","last_fullsized":"2024-03-21T06:38:58.483335+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.530393+0000","last_change":"2024-03-21T06:25:58.500230+0000","last_active":"2024-03-21T06:38:58.530393+0000","last_peered":"2024-03-21T06:38:58.530393+0000","last_clean":"2024-03-21T06:38:58.530393+0000","last_became_active":"2024-03-21T06:25:58.488027+0000","last_became_peered":"2024-03-21T06:25:58.488027+0000","last_unstale":"2024-03-21T06:38:58.530393+0000","last_undegraded":"2024-03-21T06:38:58.530393+0000","last_fullsized":"2024-03-21T06:38:58.530393+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.488223+0000","last_change":"2024-03-21T06:25:58.505580+0000","last_active":"2024-03-21T06:38:58.488223+0000","last_peered":"2024-03-21T06:38:58.488223+0000","last_clean":"2024-03-21T06:38:58.488223+0000","last_became_active":"2024-03-21T06:25:58.498450+0000","last_became_peered":"2024-03-21T06:25:58.498450+0000","last_unstale":"2024-03-21T06:38:58.488223+0000","last_undegraded":"2024-03-21T06:38:58.488223+0000","last_fullsized":"2024-03-21T06:38:58.488223+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.720009+0000","last_change":"2024-03-21T06:25:58.515242+0000","last_active":"2024-03-21T06:38:29.720009+0000","last_peered":"2024-03-21T06:38:29.720009+0000","last_clean":"2024-03-21T06:38:29.720009+0000","last_became_active":"2024-03-21T06:25:58.503376+0000","last_became_peered":"2024-03-21T06:25:58.503376+0000","last_unstale":"2024-03-21T06:38:29.720009+0000","last_undegraded":"2024-03-21T06:38:29.720009+0000","last_fullsized":"2024-03-21T06:38:29.720009+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.517006+0000","last_change":"2024-03-21T06:25:58.500753+0000","last_active":"2024-03-21T06:38:27.517006+0000","last_peered":"2024-03-21T06:38:27.517006+0000","last_clean":"2024-03-21T06:38:27.517006+0000","last_became_active":"2024-03-21T06:25:58.497498+0000","last_became_peered":"2024-03-21T06:25:58.497498+0000","last_unstale":"2024-03-21T06:38:27.517006+0000","last_undegraded":"2024-03-21T06:38:27.517006+0000","last_fullsized":"2024-03-21T06:38:27.517006+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.695561+0000","last_change":"2024-03-21T06:25:58.509601+0000","last_active":"2024-03-21T06:38:29.695561+0000","last_peered":"2024-03-21T06:38:29.695561+0000","last_clean":"2024-03-21T06:38:29.695561+0000","last_became_active":"2024-03-21T06:25:58.498895+0000","last_became_peered":"2024-03-21T06:25:58.498895+0000","last_unstale":"2024-03-21T06:38:29.695561+0000","last_undegraded":"2024-03-21T06:38:29.695561+0000","last_fullsized":"2024-03-21T06:38:29.695561+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.505272+0000","last_change":"2024-03-21T06:25:58.496259+0000","last_active":"2024-03-21T06:38:58.505272+0000","last_peered":"2024-03-21T06:38:58.505272+0000","last_clean":"2024-03-21T06:38:58.505272+0000","last_became_active":"2024-03-21T06:25:58.486238+0000","last_became_peered":"2024-03-21T06:25:58.486238+0000","last_unstale":"2024-03-21T06:38:58.505272+0000","last_undegraded":"2024-03-21T06:38:58.505272+0000","last_fullsized":"2024-03-21T06:38:58.505272+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.711452+0000","last_change":"2024-03-21T06:25:58.524275+0000","last_active":"2024-03-21T06:38:29.711452+0000","last_peered":"2024-03-21T06:38:29.711452+0000","last_clean":"2024-03-21T06:38:29.711452+0000","last_became_active":"2024-03-21T06:25:58.513806+0000","last_became_peered":"2024-03-21T06:25:58.513806+0000","last_unstale":"2024-03-21T06:38:29.711452+0000","last_undegraded":"2024-03-21T06:38:29.711452+0000","last_fullsized":"2024-03-21T06:38:29.711452+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.707100+0000","last_change":"2024-03-21T06:25:58.540399+0000","last_active":"2024-03-21T06:38:29.707100+0000","last_peered":"2024-03-21T06:38:29.707100+0000","last_clean":"2024-03-21T06:38:29.707100+0000","last_became_active":"2024-03-21T06:25:58.537009+0000","last_became_peered":"2024-03-21T06:25:58.537009+0000","last_unstale":"2024-03-21T06:38:29.707100+0000","last_undegraded":"2024-03-21T06:38:29.707100+0000","last_fullsized":"2024-03-21T06:38:29.707100+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.482596+0000","last_change":"2024-03-21T06:25:58.514471+0000","last_active":"2024-03-21T06:38:58.482596+0000","last_peered":"2024-03-21T06:38:58.482596+0000","last_clean":"2024-03-21T06:38:58.482596+0000","last_became_active":"2024-03-21T06:25:58.510651+0000","last_became_peered":"2024-03-21T06:25:58.510651+0000","last_unstale":"2024-03-21T06:38:58.482596+0000","last_undegraded":"2024-03-21T06:38:58.482596+0000","last_fullsized":"2024-03-21T06:38:58.482596+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.529785+0000","last_change":"2024-03-21T06:25:58.551761+0000","last_active":"2024-03-21T06:38:58.529785+0000","last_peered":"2024-03-21T06:38:58.529785+0000","last_clean":"2024-03-21T06:38:58.529785+0000","last_became_active":"2024-03-21T06:25:58.548956+0000","last_became_peered":"2024-03-21T06:25:58.548956+0000","last_unstale":"2024-03-21T06:38:58.529785+0000","last_undegraded":"2024-03-21T06:38:58.529785+0000","last_fullsized":"2024-03-21T06:38:58.529785+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.499863+0000","last_change":"2024-03-21T06:25:58.490344+0000","last_active":"2024-03-21T06:38:58.499863+0000","last_peered":"2024-03-21T06:38:58.499863+0000","last_clean":"2024-03-21T06:38:58.499863+0000","last_became_active":"2024-03-21T06:25:58.482358+0000","last_became_peered":"2024-03-21T06:25:58.482358+0000","last_unstale":"2024-03-21T06:38:58.499863+0000","last_undegraded":"2024-03-21T06:38:58.499863+0000","last_fullsized":"2024-03-21T06:38:58.499863+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.718502+0000","last_change":"2024-03-21T06:25:58.527681+0000","last_active":"2024-03-21T06:38:29.718502+0000","last_peered":"2024-03-21T06:38:29.718502+0000","last_clean":"2024-03-21T06:38:29.718502+0000","last_became_active":"2024-03-21T06:25:58.521822+0000","last_became_peered":"2024-03-21T06:25:58.521822+0000","last_unstale":"2024-03-21T06:38:29.718502+0000","last_undegraded":"2024-03-21T06:38:29.718502+0000","last_fullsized":"2024-03-21T06:38:29.718502+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.526661+0000","last_change":"2024-03-21T06:25:58.490537+0000","last_active":"2024-03-21T06:38:27.526661+0000","last_peered":"2024-03-21T06:38:27.526661+0000","last_clean":"2024-03-21T06:38:27.526661+0000","last_became_active":"2024-03-21T06:25:58.484498+0000","last_became_peered":"2024-03-21T06:25:58.484498+0000","last_unstale":"2024-03-21T06:38:27.526661+0000","last_undegraded":"2024-03-21T06:38:27.526661+0000","last_fullsized":"2024-03-21T06:38:27.526661+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.734123+0000","last_change":"2024-03-21T06:25:58.482707+0000","last_active":"2024-03-21T06:38:29.734123+0000","last_peered":"2024-03-21T06:38:29.734123+0000","last_clean":"2024-03-21T06:38:29.734123+0000","last_became_active":"2024-03-21T06:25:58.474376+0000","last_became_peered":"2024-03-21T06:25:58.474376+0000","last_unstale":"2024-03-21T06:38:29.734123+0000","last_undegraded":"2024-03-21T06:38:29.734123+0000","last_fullsized":"2024-03-21T06:38:29.734123+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.754301+0000","last_change":"2024-03-21T06:25:58.483801+0000","last_active":"2024-03-21T06:38:29.754301+0000","last_peered":"2024-03-21T06:38:29.754301+0000","last_clean":"2024-03-21T06:38:29.754301+0000","last_became_active":"2024-03-21T06:25:58.477107+0000","last_became_peered":"2024-03-21T06:25:58.477107+0000","last_unstale":"2024-03-21T06:38:29.754301+0000","last_undegraded":"2024-03-21T06:38:29.754301+0000","last_fullsized":"2024-03-21T06:38:29.754301+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.736832+0000","last_change":"2024-03-21T06:25:58.494316+0000","last_active":"2024-03-21T06:38:29.736832+0000","last_peered":"2024-03-21T06:38:29.736832+0000","last_clean":"2024-03-21T06:38:29.736832+0000","last_became_active":"2024-03-21T06:25:58.490894+0000","last_became_peered":"2024-03-21T06:25:58.490894+0000","last_unstale":"2024-03-21T06:38:29.736832+0000","last_undegraded":"2024-03-21T06:38:29.736832+0000","last_fullsized":"2024-03-21T06:38:29.736832+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.523400+0000","last_change":"2024-03-21T06:25:58.508102+0000","last_active":"2024-03-21T06:38:27.523400+0000","last_peered":"2024-03-21T06:38:27.523400+0000","last_clean":"2024-03-21T06:38:27.523400+0000","last_became_active":"2024-03-21T06:25:58.505636+0000","last_became_peered":"2024-03-21T06:25:58.505636+0000","last_unstale":"2024-03-21T06:38:27.523400+0000","last_undegraded":"2024-03-21T06:38:27.523400+0000","last_fullsized":"2024-03-21T06:38:27.523400+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.714562+0000","last_change":"2024-03-21T06:25:58.512176+0000","last_active":"2024-03-21T06:38:29.714562+0000","last_peered":"2024-03-21T06:38:29.714562+0000","last_clean":"2024-03-21T06:38:29.714562+0000","last_became_active":"2024-03-21T06:25:58.501608+0000","last_became_peered":"2024-03-21T06:25:58.501608+0000","last_unstale":"2024-03-21T06:38:29.714562+0000","last_undegraded":"2024-03-21T06:38:29.714562+0000","last_fullsized":"2024-03-21T06:38:29.714562+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.715197+0000","last_change":"2024-03-21T06:25:58.517908+0000","last_active":"2024-03-21T06:38:29.715197+0000","last_peered":"2024-03-21T06:38:29.715197+0000","last_clean":"2024-03-21T06:38:29.715197+0000","last_became_active":"2024-03-21T06:25:58.508192+0000","last_became_peered":"2024-03-21T06:25:58.508192+0000","last_unstale":"2024-03-21T06:38:29.715197+0000","last_undegraded":"2024-03-21T06:38:29.715197+0000","last_fullsized":"2024-03-21T06:38:29.715197+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.517104+0000","last_change":"2024-03-21T06:25:58.501302+0000","last_active":"2024-03-21T06:38:58.517104+0000","last_peered":"2024-03-21T06:38:58.517104+0000","last_clean":"2024-03-21T06:38:58.517104+0000","last_became_active":"2024-03-21T06:25:58.491525+0000","last_became_peered":"2024-03-21T06:25:58.491525+0000","last_unstale":"2024-03-21T06:38:58.517104+0000","last_undegraded":"2024-03-21T06:38:58.517104+0000","last_fullsized":"2024-03-21T06:38:58.517104+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.700998+0000","last_change":"2024-03-21T06:25:58.516179+0000","last_active":"2024-03-21T06:38:29.700998+0000","last_peered":"2024-03-21T06:38:29.700998+0000","last_clean":"2024-03-21T06:38:29.700998+0000","last_became_active":"2024-03-21T06:25:58.506657+0000","last_became_peered":"2024-03-21T06:25:58.506657+0000","last_unstale":"2024-03-21T06:38:29.700998+0000","last_undegraded":"2024-03-21T06:38:29.700998+0000","last_fullsized":"2024-03-21T06:38:29.700998+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.490947+0000","last_change":"2024-03-21T06:25:58.521034+0000","last_active":"2024-03-21T06:38:58.490947+0000","last_peered":"2024-03-21T06:38:58.490947+0000","last_clean":"2024-03-21T06:38:58.490947+0000","last_became_active":"2024-03-21T06:25:58.517030+0000","last_became_peered":"2024-03-21T06:25:58.517030+0000","last_unstale":"2024-03-21T06:38:58.490947+0000","last_undegraded":"2024-03-21T06:38:58.490947+0000","last_fullsized":"2024-03-21T06:38:58.490947+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.493691+0000","last_change":"2024-03-21T06:25:58.531640+0000","last_active":"2024-03-21T06:38:58.493691+0000","last_peered":"2024-03-21T06:38:58.493691+0000","last_clean":"2024-03-21T06:38:58.493691+0000","last_became_active":"2024-03-21T06:25:58.528601+0000","last_became_peered":"2024-03-21T06:25:58.528601+0000","last_unstale":"2024-03-21T06:38:58.493691+0000","last_undegraded":"2024-03-21T06:38:58.493691+0000","last_fullsized":"2024-03-21T06:38:58.493691+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.532701+0000","last_change":"2024-03-21T06:25:58.509341+0000","last_active":"2024-03-21T06:38:58.532701+0000","last_peered":"2024-03-21T06:38:58.532701+0000","last_clean":"2024-03-21T06:38:58.532701+0000","last_became_active":"2024-03-21T06:25:58.503544+0000","last_became_peered":"2024-03-21T06:25:58.503544+0000","last_unstale":"2024-03-21T06:38:58.532701+0000","last_undegraded":"2024-03-21T06:38:58.532701+0000","last_fullsized":"2024-03-21T06:38:58.532701+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"236'7","reported_seq":1284,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.757963+0000","last_change":"2024-03-21T06:25:58.509187+0000","last_active":"2024-03-21T06:38:29.757963+0000","last_peered":"2024-03-21T06:38:29.757963+0000","last_clean":"2024-03-21T06:38:29.757963+0000","last_became_active":"2024-03-21T06:25:58.505585+0000","last_became_peered":"2024-03-21T06:25:58.505585+0000","last_unstale":"2024-03-21T06:38:29.757963+0000","last_undegraded":"2024-03-21T06:38:29.757963+0000","last_fullsized":"2024-03-21T06:38:29.757963+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.513968+0000","last_change":"2024-03-21T06:25:58.486325+0000","last_active":"2024-03-21T06:38:27.513968+0000","last_peered":"2024-03-21T06:38:27.513968+0000","last_clean":"2024-03-21T06:38:27.513968+0000","last_became_active":"2024-03-21T06:25:58.481996+0000","last_became_peered":"2024-03-21T06:25:58.481996+0000","last_unstale":"2024-03-21T06:38:27.513968+0000","last_undegraded":"2024-03-21T06:38:27.513968+0000","last_fullsized":"2024-03-21T06:38:27.513968+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1870,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.736177+0000","last_change":"2024-03-21T06:20:23.724343+0000","last_active":"2024-03-21T06:38:29.736177+0000","last_peered":"2024-03-21T06:38:29.736177+0000","last_clean":"2024-03-21T06:38:29.736177+0000","last_became_active":"2024-03-21T06:20:23.721083+0000","last_became_peered":"2024-03-21T06:20:23.721083+0000","last_unstale":"2024-03-21T06:38:29.736177+0000","last_undegraded":"2024-03-21T06:38:29.736177+0000","last_fullsized":"2024-03-21T06:38:29.736177+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1880,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.608992+0000","last_change":"2024-03-21T06:20:23.720094+0000","last_active":"2024-03-21T06:38:27.608992+0000","last_peered":"2024-03-21T06:38:27.608992+0000","last_clean":"2024-03-21T06:38:27.608992+0000","last_became_active":"2024-03-21T06:20:23.717296+0000","last_became_peered":"2024-03-21T06:20:23.717296+0000","last_unstale":"2024-03-21T06:38:27.608992+0000","last_undegraded":"2024-03-21T06:38:27.608992+0000","last_fullsized":"2024-03-21T06:38:27.608992+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1864,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.785694+0000","last_change":"2024-03-21T06:20:23.724647+0000","last_active":"2024-03-21T06:38:29.785694+0000","last_peered":"2024-03-21T06:38:29.785694+0000","last_clean":"2024-03-21T06:38:29.785694+0000","last_became_active":"2024-03-21T06:20:23.721198+0000","last_became_peered":"2024-03-21T06:20:23.721198+0000","last_unstale":"2024-03-21T06:38:29.785694+0000","last_undegraded":"2024-03-21T06:38:29.785694+0000","last_fullsized":"2024-03-21T06:38:29.785694+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1880,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.522321+0000","last_change":"2024-03-21T06:20:23.716953+0000","last_active":"2024-03-21T06:38:27.522321+0000","last_peered":"2024-03-21T06:38:27.522321+0000","last_clean":"2024-03-21T06:38:27.522321+0000","last_became_active":"2024-03-21T06:20:23.714274+0000","last_became_peered":"2024-03-21T06:20:23.714274+0000","last_unstale":"2024-03-21T06:38:27.522321+0000","last_undegraded":"2024-03-21T06:38:27.522321+0000","last_fullsized":"2024-03-21T06:38:27.522321+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1867,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.546801+0000","last_change":"2024-03-21T06:20:23.717551+0000","last_active":"2024-03-21T06:38:58.546801+0000","last_peered":"2024-03-21T06:38:58.546801+0000","last_clean":"2024-03-21T06:38:58.546801+0000","last_became_active":"2024-03-21T06:20:23.714766+0000","last_became_peered":"2024-03-21T06:20:23.714766+0000","last_unstale":"2024-03-21T06:38:58.546801+0000","last_undegraded":"2024-03-21T06:38:58.546801+0000","last_fullsized":"2024-03-21T06:38:58.546801+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1870,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.759598+0000","last_change":"2024-03-21T06:20:23.716172+0000","last_active":"2024-03-21T06:38:29.759598+0000","last_peered":"2024-03-21T06:38:29.759598+0000","last_clean":"2024-03-21T06:38:29.759598+0000","last_became_active":"2024-03-21T06:20:23.712781+0000","last_became_peered":"2024-03-21T06:20:23.712781+0000","last_unstale":"2024-03-21T06:38:29.759598+0000","last_undegraded":"2024-03-21T06:38:29.759598+0000","last_fullsized":"2024-03-21T06:38:29.759598+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1867,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.557390+0000","last_change":"2024-03-21T06:20:23.724311+0000","last_active":"2024-03-21T06:38:58.557390+0000","last_peered":"2024-03-21T06:38:58.557390+0000","last_clean":"2024-03-21T06:38:58.557390+0000","last_became_active":"2024-03-21T06:20:23.720894+0000","last_became_peered":"2024-03-21T06:20:23.720894+0000","last_unstale":"2024-03-21T06:38:58.557390+0000","last_undegraded":"2024-03-21T06:38:58.557390+0000","last_fullsized":"2024-03-21T06:38:58.557390+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1870,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.809884+0000","last_change":"2024-03-21T06:20:23.720175+0000","last_active":"2024-03-21T06:38:29.809884+0000","last_peered":"2024-03-21T06:38:29.809884+0000","last_clean":"2024-03-21T06:38:29.809884+0000","last_became_active":"2024-03-21T06:20:23.716768+0000","last_became_peered":"2024-03-21T06:20:23.716768+0000","last_unstale":"2024-03-21T06:38:29.809884+0000","last_undegraded":"2024-03-21T06:38:29.809884+0000","last_fullsized":"2024-03-21T06:38:29.809884+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":1997,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.703106+0000","last_change":"2024-03-21T06:20:21.697506+0000","last_active":"2024-03-21T06:38:29.703106+0000","last_peered":"2024-03-21T06:38:29.703106+0000","last_clean":"2024-03-21T06:38:29.703106+0000","last_became_active":"2024-03-21T06:20:21.690580+0000","last_became_peered":"2024-03-21T06:20:21.690580+0000","last_unstale":"2024-03-21T06:38:29.703106+0000","last_undegraded":"2024-03-21T06:38:29.703106+0000","last_fullsized":"2024-03-21T06:38:29.703106+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:20.658895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:20.658895+0000","last_clean_scrub_stamp":"2024-03-21T06:20:20.658895+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":71252,"kb_used_data":4044,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":104786348,"statfs":{"total":107374182400,"available":107301220352,"internally_reserved":0,"allocated":4141056,"data_stored":2006883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":70756,"kb_used_data":3548,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":104786844,"statfs":{"total":107374182400,"available":107301728256,"internally_reserved":0,"allocated":3633152,"data_stored":1449822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":70740,"kb_used_data":3532,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":104786860,"statfs":{"total":107374182400,"available":107301744640,"internally_reserved":0,"allocated":3616768,"data_stored":1448988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":75484,"kb_used_data":4116,"kb_used_omap":0,"kb_used_meta":71360,"kb_avail":104782116,"statfs":{"total":107374182400,"available":107296886784,"internally_reserved":0,"allocated":4214784,"data_stored":2051132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":73072640},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-21T06:39:01.101 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T06:39:01.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.203+0000 7f4e2c66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:01.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.207+0000 7f4e2c66d640 1 Processor -- start 2024-03-21T06:39:01.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.207+0000 7f4e2c66d640 1 -- start start 2024-03-21T06:39:01.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.208+0000 7f4e2c66d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4e2409cda0 0x7f4e24139450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:01.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.208+0000 7f4e2c66d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e2406e410 con 0x7f4e2409b360 2024-03-21T06:39:01.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.208+0000 7f4e2c66d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e2409b770 con 0x7f4e2409cda0 2024-03-21T06:39:01.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.208+0000 7f4e29c08640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4e2409cda0 0x7f4e24139450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:01.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.208+0000 7f4e2a409640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f4e2409b360 0x7f4e241315d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:53126/0 (socket says 172.21.15.96:53126) 2024-03-21T06:39:01.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.208+0000 7f4e2a409640 1 -- 172.21.15.96:0/2259446956 learned_addr learned my addr 172.21.15.96:0/2259446956 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:39:01.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.208+0000 7f4e29c08640 1 -- 172.21.15.96:0/2259446956 >> v1:172.21.15.96:6789/0 conn(0x7f4e2409b360 legacy=0x7f4e241315d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:01.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.209+0000 7f4e29407640 1 -- 172.21.15.96:0/2259446956 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148662398 0 0) 0x7f4e2406e410 con 0x7f4e2409b360 2024-03-21T06:39:01.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.209+0000 7f4e29c08640 1 -- 172.21.15.96:0/2259446956 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2409d1b0 con 0x7f4e2409cda0 2024-03-21T06:39:01.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.209+0000 7f4e29c08640 1 --2- 172.21.15.96:0/2259446956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e2409cda0 0x7f4e24139450 crc :-1 s=READY pgs=12989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0a8186b152ea02 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:01.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.209+0000 7f4e29407640 1 -- 172.21.15.96:0/2259446956 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4e20009ce0 con 0x7f4e2409cda0 2024-03-21T06:39:01.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.209+0000 7f4e29407640 1 -- 172.21.15.96:0/2259446956 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4e20009e40 con 0x7f4e2409cda0 2024-03-21T06:39:01.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.209+0000 7f4e29407640 1 -- 172.21.15.96:0/2259446956 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4e2000e360 con 0x7f4e2409cda0 2024-03-21T06:39:01.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.209+0000 7f4e2c66d640 1 -- 172.21.15.96:0/2259446956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e2409cda0 msgr2=0x7f4e24139450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:01.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.209+0000 7f4e2c66d640 1 --2- 172.21.15.96:0/2259446956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e2409cda0 0x7f4e24139450 crc :-1 s=READY pgs=12989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:01.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.210+0000 7f4e2c66d640 1 -- 172.21.15.96:0/2259446956 shutdown_connections 2024-03-21T06:39:01.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.210+0000 7f4e2c66d640 1 --2- 172.21.15.96:0/2259446956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e2409cda0 0x7f4e24139450 unknown :-1 s=CLOSED pgs=12989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:01.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.210+0000 7f4e2c66d640 1 -- 172.21.15.96:0/2259446956 >> 172.21.15.96:0/2259446956 conn(0x7f4e240a42c0 msgr2=0x7f4e240a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:01.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.210+0000 7f4e2c66d640 1 -- 172.21.15.96:0/2259446956 shutdown_connections 2024-03-21T06:39:01.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.210+0000 7f4e2c66d640 1 -- 172.21.15.96:0/2259446956 wait complete. 2024-03-21T06:39:01.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.210+0000 7f4e2c66d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:01.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.211+0000 7f4e2c66d640 1 Processor -- start 2024-03-21T06:39:01.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.211+0000 7f4e2c66d640 1 -- start start 2024-03-21T06:39:01.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.211+0000 7f4e2c66d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e2409b360 0x7f4e2411d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:01.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.211+0000 7f4e2c66d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e241384d0 con 0x7f4e2409b360 2024-03-21T06:39:01.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.211+0000 7f4e2a409640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e2409b360 0x7f4e2411d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:01.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.211+0000 7f4e2a409640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e2409b360 0x7f4e2411d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55930/0 (socket says 172.21.15.96:55930) 2024-03-21T06:39:01.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.211+0000 7f4e2a409640 1 -- 172.21.15.96:0/1368666772 learned_addr learned my addr 172.21.15.96:0/1368666772 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:39:01.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.212+0000 7f4e2a409640 1 -- 172.21.15.96:0/1368666772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2411ca10 con 0x7f4e2409b360 2024-03-21T06:39:01.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.212+0000 7f4e2a409640 1 --2- 172.21.15.96:0/1368666772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e2409b360 0x7f4e2411d070 crc :-1 s=READY pgs=12990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:01.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.212+0000 7f4e1affd640 1 -- 172.21.15.96:0/1368666772 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4e14004af0 con 0x7f4e2409b360 2024-03-21T06:39:01.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.212+0000 7f4e1affd640 1 -- 172.21.15.96:0/1368666772 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f4e14004c50 con 0x7f4e2409b360 2024-03-21T06:39:01.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.212+0000 7f4e1affd640 1 -- 172.21.15.96:0/1368666772 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f4e14007120 con 0x7f4e2409b360 2024-03-21T06:39:01.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.212+0000 7f4e2c66d640 1 -- 172.21.15.96:0/1368666772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2411dd90 con 0x7f4e2409b360 2024-03-21T06:39:01.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.212+0000 7f4e2c66d640 1 -- 172.21.15.96:0/1368666772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2411f580 con 0x7f4e2409b360 2024-03-21T06:39:01.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.213+0000 7f4e1affd640 1 -- 172.21.15.96:0/1368666772 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f4e140072d0 con 0x7f4e2409b360 2024-03-21T06:39:01.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.213+0000 7f4e2c66d640 1 -- 172.21.15.96:0/1368666772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e2411df20 con 0x7f4e2409b360 2024-03-21T06:39:01.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.213+0000 7f4e1affd640 1 --2- 172.21.15.96:0/1368666772 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4e08038260 0x7f4e0803a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:01.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.213+0000 7f4e1affd640 1 -- 172.21.15.96:0/1368666772 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(713..713 src has 402..713) v4 ==== 3403+0+0 (crc 0 0 0) 0x7f4e14047d00 con 0x7f4e2409b360 2024-03-21T06:39:01.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.213+0000 7f4e29c08640 1 --2- 172.21.15.96:0/1368666772 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4e08038260 0x7f4e0803a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:01.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.214+0000 7f4e29c08640 1 --2- 172.21.15.96:0/1368666772 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4e08038260 0x7f4e0803a770 crc :-1 s=READY pgs=3865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:01.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.218+0000 7f4e1affd640 1 -- 172.21.15.96:0/1368666772 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f4e1400a9e0 con 0x7f4e2409b360 2024-03-21T06:39:01.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.391+0000 7f4e2c66d640 1 -- 172.21.15.96:0/1368666772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e240a2630 con 0x7f4e2409b360 2024-03-21T06:39:01.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.392+0000 7f4e1affd640 1 -- 172.21.15.96:0/1368666772 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v713) v1 ==== 74+0+11103 (crc 0 0 0) 0x7f4e1400a5d0 con 0x7f4e2409b360 2024-03-21T06:39:01.391 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:39:01.391 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":713,"fsid":"e880df84-2d7c-4386-b5e9-f3879b9ab9d5","created":"2024-03-21T06:20:12.270291+0000","modified":"2024-03-21T06:38:59.474606+0000","last_up_change":"2024-03-21T06:20:18.657108+0000","last_in_change":"2024-03-21T06:20:14.428643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":201,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T06:20:20.651568+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-21T06:20:22.673961+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":75,"pool_name":"ceph_test_rados_api_asio","create_time":"2024-03-21T06:25:56.428495+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffab25be-1f6f-450d-a8b4-da7f86f3eaeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":700,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6801","nonce":932229279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":932229279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":932229279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6803","nonce":932229279}]},"public_addr":"172.21.15.96:6801/932229279","cluster_addr":"172.21.15.96:6802/932229279","heartbeat_back_addr":"172.21.15.96:6808/932229279","heartbeat_front_addr":"172.21.15.96:6803/932229279","state":["exists","up"]},{"osd":1,"uuid":"3266cd80-6286-4a69-933a-00cd48d67487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":700,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6805","nonce":13988474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":13988474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":13988474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6807","nonce":13988474}]},"public_addr":"172.21.15.96:6805/13988474","cluster_addr":"172.21.15.96:6804/13988474","heartbeat_back_addr":"172.21.15.96:6806/13988474","heartbeat_front_addr":"172.21.15.96:6807/13988474","state":["exists","up"]},{"osd":2,"uuid":"5a8b1e5a-d4ed-4c96-b301-4173028235d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":700,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":155456265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6805","nonce":155456265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":155456265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6815","nonce":155456265}]},"public_addr":"172.21.15.184:6804/155456265","cluster_addr":"172.21.15.184:6805/155456265","heartbeat_back_addr":"172.21.15.184:6814/155456265","heartbeat_front_addr":"172.21.15.184:6815/155456265","state":["exists","up"]},{"osd":3,"uuid":"5945ad82-8437-4490-aca6-9c454678cd34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":700,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6811","nonce":276480837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":276480837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":276480837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6809","nonce":276480837}]},"public_addr":"172.21.15.184:6811/276480837","cluster_addr":"172.21.15.184:6810/276480837","heartbeat_back_addr":"172.21.15.184:6808/276480837","heartbeat_front_addr":"172.21.15.184:6809/276480837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":100,"snaps":[{"begin":1,"length":4}]},{"pool":101,"snaps":[{"begin":2,"length":4},{"begin":7,"length":2}]},{"pool":102,"snaps":[{"begin":1,"length":6}]},{"pool":103,"snaps":[{"begin":2,"length":4},{"begin":9,"length":1}]},{"pool":104,"snaps":[{"begin":2,"length":1}]},{"pool":180,"snaps":[{"begin":1,"length":1}]},{"pool":181,"snaps":[{"begin":1,"length":1}]},{"pool":183,"snaps":[{"begin":1,"length":1}]},{"pool":184,"snaps":[{"begin":1,"length":2}]},{"pool":185,"snaps":[{"begin":3,"length":1}]},{"pool":188,"snaps":[{"begin":2,"length":2}]},{"pool":190,"snaps":[{"begin":3,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T06:39:01.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.394+0000 7f4e2c66d640 1 -- 172.21.15.96:0/1368666772 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4e08038260 msgr2=0x7f4e0803a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:01.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.394+0000 7f4e2c66d640 1 --2- 172.21.15.96:0/1368666772 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4e08038260 0x7f4e0803a770 crc :-1 s=READY pgs=3865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:01.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.394+0000 7f4e2c66d640 1 -- 172.21.15.96:0/1368666772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e2409b360 msgr2=0x7f4e2411d070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:01.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.394+0000 7f4e2c66d640 1 --2- 172.21.15.96:0/1368666772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e2409b360 0x7f4e2411d070 crc :-1 s=READY pgs=12990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:01.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.394+0000 7f4e2c66d640 1 -- 172.21.15.96:0/1368666772 shutdown_connections 2024-03-21T06:39:01.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.394+0000 7f4e2c66d640 1 --2- 172.21.15.96:0/1368666772 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f4e08038260 0x7f4e0803a770 unknown :-1 s=CLOSED pgs=3865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:01.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.394+0000 7f4e2c66d640 1 --2- 172.21.15.96:0/1368666772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e2409b360 0x7f4e2411d070 unknown :-1 s=CLOSED pgs=12990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:01.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.394+0000 7f4e2c66d640 1 -- 172.21.15.96:0/1368666772 >> 172.21.15.96:0/1368666772 conn(0x7f4e240a42c0 msgr2=0x7f4e240a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:01.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.394+0000 7f4e2c66d640 1 -- 172.21.15.96:0/1368666772 shutdown_connections 2024-03-21T06:39:01.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:01.394+0000 7f4e2c66d640 1 -- 172.21.15.96:0/1368666772 wait complete. 2024-03-21T06:39:02.405 INFO:tasks.ceph:Scrubbing osd.0 2024-03-21T06:39:02.405 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-21T06:39:02.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.523+0000 7fe29f42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:02.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.531+0000 7fe29f42d640 1 Processor -- start 2024-03-21T06:39:02.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.532+0000 7fe29f42d640 1 -- start start 2024-03-21T06:39:02.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.532+0000 7fe29f42d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe29809cda0 0x7fe298139450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:02.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.532+0000 7fe29f42d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29806e410 con 0x7fe29809b360 2024-03-21T06:39:02.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.532+0000 7fe29f42d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe29809b770 con 0x7fe29809cda0 2024-03-21T06:39:02.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.532+0000 7fe29c9c8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe29809cda0 0x7fe298139450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:02.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.532+0000 7fe29d1c9640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fe29809b360 0x7fe2981315d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:53142/0 (socket says 172.21.15.96:53142) 2024-03-21T06:39:02.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.532+0000 7fe29d1c9640 1 -- 172.21.15.96:0/2515490479 learned_addr learned my addr 172.21.15.96:0/2515490479 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:39:02.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.533+0000 7fe29c9c8640 1 -- 172.21.15.96:0/2515490479 >> v1:172.21.15.96:6789/0 conn(0x7fe29809b360 legacy=0x7fe2981315d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:02.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.533+0000 7fe29c9c8640 1 -- 172.21.15.96:0/2515490479 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29809d1b0 con 0x7fe29809cda0 2024-03-21T06:39:02.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.533+0000 7fe29c9c8640 1 --2- 172.21.15.96:0/2515490479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe29809cda0 0x7fe298139450 crc :-1 s=READY pgs=12992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c954e41b23471fcf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:02.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.533+0000 7fe28ffff640 1 -- 172.21.15.96:0/2515490479 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe288009ce0 con 0x7fe29809cda0 2024-03-21T06:39:02.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.533+0000 7fe28ffff640 1 -- 172.21.15.96:0/2515490479 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe288009e40 con 0x7fe29809cda0 2024-03-21T06:39:02.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.533+0000 7fe28ffff640 1 -- 172.21.15.96:0/2515490479 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe28800e360 con 0x7fe29809cda0 2024-03-21T06:39:02.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.534+0000 7fe29f42d640 1 -- 172.21.15.96:0/2515490479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe29809cda0 msgr2=0x7fe298139450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:02.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.534+0000 7fe29f42d640 1 --2- 172.21.15.96:0/2515490479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe29809cda0 0x7fe298139450 crc :-1 s=READY pgs=12992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:02.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.534+0000 7fe29f42d640 1 -- 172.21.15.96:0/2515490479 shutdown_connections 2024-03-21T06:39:02.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.534+0000 7fe29f42d640 1 --2- 172.21.15.96:0/2515490479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe29809cda0 0x7fe298139450 unknown :-1 s=CLOSED pgs=12992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:02.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.534+0000 7fe29f42d640 1 -- 172.21.15.96:0/2515490479 >> 172.21.15.96:0/2515490479 conn(0x7fe2980a42c0 msgr2=0x7fe2980a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:02.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.534+0000 7fe29f42d640 1 -- 172.21.15.96:0/2515490479 shutdown_connections 2024-03-21T06:39:02.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.534+0000 7fe29f42d640 1 -- 172.21.15.96:0/2515490479 wait complete. 2024-03-21T06:39:02.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.534+0000 7fe29f42d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:02.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.535+0000 7fe29f42d640 1 Processor -- start 2024-03-21T06:39:02.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.535+0000 7fe29f42d640 1 -- start start 2024-03-21T06:39:02.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.536+0000 7fe29f42d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe29809b360 0x7fe2981c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:02.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.536+0000 7fe29f42d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29811e420 con 0x7fe29809b360 2024-03-21T06:39:02.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.536+0000 7fe29d1c9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe29809b360 0x7fe2981c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:02.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.536+0000 7fe29d1c9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe29809b360 0x7fe2981c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55954/0 (socket says 172.21.15.96:55954) 2024-03-21T06:39:02.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.536+0000 7fe29d1c9640 1 -- 172.21.15.96:0/2759213598 learned_addr learned my addr 172.21.15.96:0/2759213598 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:39:02.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.536+0000 7fe29d1c9640 1 -- 172.21.15.96:0/2759213598 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe298073030 con 0x7fe29809b360 2024-03-21T06:39:02.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.536+0000 7fe29d1c9640 1 --2- 172.21.15.96:0/2759213598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe29809b360 0x7fe2981c8d20 crc :-1 s=READY pgs=12993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:02.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.536+0000 7fe28dffb640 1 -- 172.21.15.96:0/2759213598 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe2940070b0 con 0x7fe29809b360 2024-03-21T06:39:02.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.536+0000 7fe28dffb640 1 -- 172.21.15.96:0/2759213598 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fe294007210 con 0x7fe29809b360 2024-03-21T06:39:02.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.536+0000 7fe29f42d640 1 -- 172.21.15.96:0/2759213598 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29811cd30 con 0x7fe29809b360 2024-03-21T06:39:02.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.536+0000 7fe28dffb640 1 -- 172.21.15.96:0/2759213598 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fe2940074b0 con 0x7fe29809b360 2024-03-21T06:39:02.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.537+0000 7fe29f42d640 1 -- 172.21.15.96:0/2759213598 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29811d510 con 0x7fe29809b360 2024-03-21T06:39:02.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.537+0000 7fe28dffb640 1 -- 172.21.15.96:0/2759213598 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fe2940076a0 con 0x7fe29809b360 2024-03-21T06:39:02.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.537+0000 7fe28dffb640 1 --2- 172.21.15.96:0/2759213598 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fe27c038260 0x7fe27c03a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:02.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.537+0000 7fe28dffb640 1 -- 172.21.15.96:0/2759213598 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(713..713 src has 402..713) v4 ==== 3403+0+0 (crc 0 0 0) 0x7fe294048420 con 0x7fe29809b360 2024-03-21T06:39:02.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.537+0000 7fe29f42d640 1 --2- 172.21.15.96:0/2759213598 >> v2:172.21.15.96:6801/932229279 conn(0x7fe29809cda0 0x7fe2981c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:02.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.537+0000 7fe29f42d640 1 -- 172.21.15.96:0/2759213598 --> v2:172.21.15.96:6801/932229279 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe2981312c0 con 0x7fe29809cda0 2024-03-21T06:39:02.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.538+0000 7fe29c9c8640 1 --2- 172.21.15.96:0/2759213598 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fe27c038260 0x7fe27c03a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:02.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.538+0000 7fe29d9ca640 1 --2- 172.21.15.96:0/2759213598 >> v2:172.21.15.96:6801/932229279 conn(0x7fe29809cda0 0x7fe2981c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:39:02.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.538+0000 7fe29c9c8640 1 --2- 172.21.15.96:0/2759213598 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fe27c038260 0x7fe27c03a770 crc :-1 s=READY pgs=3866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:02.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.540+0000 7fe29d9ca640 1 --2- 172.21.15.96:0/2759213598 >> v2:172.21.15.96:6801/932229279 conn(0x7fe29809cda0 0x7fe2981c9260 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:02.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.541+0000 7fe28dffb640 1 -- 172.21.15.96:0/2759213598 <== osd.0 v2:172.21.15.96:6801/932229279 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7fe290009280 con 0x7fe29809cda0 2024-03-21T06:39:02.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.544+0000 7fe29f42d640 1 -- 172.21.15.96:0/2759213598 --> v2:172.21.15.96:6801/932229279 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe298139990 con 0x7fe29809cda0 2024-03-21T06:39:02.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.545+0000 7fe28dffb640 1 -- 172.21.15.96:0/2759213598 <== osd.0 v2:172.21.15.96:6801/932229279 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+22 (crc 0 0 0) 0x7fe290009790 con 0x7fe29809cda0 2024-03-21T06:39:02.544 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-21T06:39:02.544 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "" 2024-03-21T06:39:02.545 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-21T06:39:02.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.545+0000 7fe29f42d640 1 -- 172.21.15.96:0/2759213598 >> v2:172.21.15.96:6801/932229279 conn(0x7fe29809cda0 msgr2=0x7fe2981c9260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:02.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.545+0000 7fe29f42d640 1 --2- 172.21.15.96:0/2759213598 >> v2:172.21.15.96:6801/932229279 conn(0x7fe29809cda0 0x7fe2981c9260 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:02.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.546+0000 7fe29f42d640 1 -- 172.21.15.96:0/2759213598 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fe27c038260 msgr2=0x7fe27c03a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:02.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.546+0000 7fe29f42d640 1 --2- 172.21.15.96:0/2759213598 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fe27c038260 0x7fe27c03a770 crc :-1 s=READY pgs=3866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:02.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.546+0000 7fe29f42d640 1 -- 172.21.15.96:0/2759213598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe29809b360 msgr2=0x7fe2981c8d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:02.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.546+0000 7fe29f42d640 1 --2- 172.21.15.96:0/2759213598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe29809b360 0x7fe2981c8d20 crc :-1 s=READY pgs=12993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:02.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.546+0000 7fe29f42d640 1 -- 172.21.15.96:0/2759213598 shutdown_connections 2024-03-21T06:39:02.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.546+0000 7fe29f42d640 1 --2- 172.21.15.96:0/2759213598 >> v2:172.21.15.96:6801/932229279 conn(0x7fe29809cda0 0x7fe2981c9260 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:02.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.546+0000 7fe29f42d640 1 --2- 172.21.15.96:0/2759213598 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fe27c038260 0x7fe27c03a770 unknown :-1 s=CLOSED pgs=3866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:02.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.546+0000 7fe29f42d640 1 --2- 172.21.15.96:0/2759213598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe29809b360 0x7fe2981c8d20 unknown :-1 s=CLOSED pgs=12993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:02.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.546+0000 7fe29f42d640 1 -- 172.21.15.96:0/2759213598 >> 172.21.15.96:0/2759213598 conn(0x7fe2980a42c0 msgr2=0x7fe2980a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:02.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.546+0000 7fe29f42d640 1 -- 172.21.15.96:0/2759213598 shutdown_connections 2024-03-21T06:39:02.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.546+0000 7fe29f42d640 1 -- 172.21.15.96:0/2759213598 wait complete. 2024-03-21T06:39:02.556 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-21T06:39:02.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.662+0000 7fb61860f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:02.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.667+0000 7fb61860f640 1 Processor -- start 2024-03-21T06:39:02.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.667+0000 7fb61860f640 1 -- start start 2024-03-21T06:39:02.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.668+0000 7fb61860f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb61009cda0 0x7fb610139450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:02.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.668+0000 7fb61860f640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb61006e410 con 0x7fb61009b360 2024-03-21T06:39:02.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.668+0000 7fb61860f640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61009b770 con 0x7fb61009cda0 2024-03-21T06:39:02.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.668+0000 7fb615baa640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb61009cda0 0x7fb610139450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:02.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.668+0000 7fb6163ab640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fb61009b360 0x7fb6101315d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:53158/0 (socket says 172.21.15.96:53158) 2024-03-21T06:39:02.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.668+0000 7fb6163ab640 1 -- 172.21.15.96:0/3978292408 learned_addr learned my addr 172.21.15.96:0/3978292408 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:39:02.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.669+0000 7fb615baa640 1 -- 172.21.15.96:0/3978292408 >> v1:172.21.15.96:6789/0 conn(0x7fb61009b360 legacy=0x7fb6101315d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:02.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.669+0000 7fb615baa640 1 -- 172.21.15.96:0/3978292408 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61009d1b0 con 0x7fb61009cda0 2024-03-21T06:39:02.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.669+0000 7fb6153a9640 1 -- 172.21.15.96:0/3978292408 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783594025 0 0) 0x7fb61006e410 con 0x7fb61009b360 2024-03-21T06:39:02.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.669+0000 7fb615baa640 1 --2- 172.21.15.96:0/3978292408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb61009cda0 0x7fb610139450 crc :-1 s=READY pgs=12995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d4b110d41a5906 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:02.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.669+0000 7fb6153a9640 1 -- 172.21.15.96:0/3978292408 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb600009ce0 con 0x7fb61009cda0 2024-03-21T06:39:02.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.669+0000 7fb6153a9640 1 -- 172.21.15.96:0/3978292408 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb600009e40 con 0x7fb61009cda0 2024-03-21T06:39:02.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.669+0000 7fb6153a9640 1 -- 172.21.15.96:0/3978292408 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb60000e360 con 0x7fb61009cda0 2024-03-21T06:39:02.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.669+0000 7fb61860f640 1 -- 172.21.15.96:0/3978292408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb61009cda0 msgr2=0x7fb610139450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:02.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.669+0000 7fb61860f640 1 --2- 172.21.15.96:0/3978292408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb61009cda0 0x7fb610139450 crc :-1 s=READY pgs=12995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:02.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.670+0000 7fb61860f640 1 -- 172.21.15.96:0/3978292408 shutdown_connections 2024-03-21T06:39:02.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.670+0000 7fb61860f640 1 --2- 172.21.15.96:0/3978292408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb61009cda0 0x7fb610139450 unknown :-1 s=CLOSED pgs=12995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:02.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.670+0000 7fb61860f640 1 -- 172.21.15.96:0/3978292408 >> 172.21.15.96:0/3978292408 conn(0x7fb6100a42c0 msgr2=0x7fb6100a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:02.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.670+0000 7fb61860f640 1 -- 172.21.15.96:0/3978292408 shutdown_connections 2024-03-21T06:39:02.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.670+0000 7fb61860f640 1 -- 172.21.15.96:0/3978292408 wait complete. 2024-03-21T06:39:02.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.670+0000 7fb61860f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:02.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.671+0000 7fb61860f640 1 Processor -- start 2024-03-21T06:39:02.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.671+0000 7fb61860f640 1 -- start start 2024-03-21T06:39:02.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.671+0000 7fb61860f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb61009b360 0x7fb6101387e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:02.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.671+0000 7fb61860f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb610073030 con 0x7fb61009b360 2024-03-21T06:39:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.671+0000 7fb6163ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb61009b360 0x7fb6101387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.671+0000 7fb6163ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb61009b360 0x7fb6101387e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55974/0 (socket says 172.21.15.96:55974) 2024-03-21T06:39:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.671+0000 7fb6163ab640 1 -- 172.21.15.96:0/379763936 learned_addr learned my addr 172.21.15.96:0/379763936 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:39:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.672+0000 7fb6163ab640 1 -- 172.21.15.96:0/379763936 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6101383b0 con 0x7fb61009b360 2024-03-21T06:39:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.672+0000 7fb6163ab640 1 --2- 172.21.15.96:0/379763936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb61009b360 0x7fb6101387e0 crc :-1 s=READY pgs=12996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.672+0000 7fb606ffd640 1 -- 172.21.15.96:0/379763936 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb60c004e70 con 0x7fb61009b360 2024-03-21T06:39:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.672+0000 7fb606ffd640 1 -- 172.21.15.96:0/379763936 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb60c00c870 con 0x7fb61009b360 2024-03-21T06:39:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.672+0000 7fb61860f640 1 -- 172.21.15.96:0/379763936 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61011dc20 con 0x7fb61009b360 2024-03-21T06:39:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.672+0000 7fb606ffd640 1 -- 172.21.15.96:0/379763936 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb60c00cb10 con 0x7fb61009b360 2024-03-21T06:39:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.672+0000 7fb61860f640 1 -- 172.21.15.96:0/379763936 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6101380a0 con 0x7fb61009b360 2024-03-21T06:39:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.673+0000 7fb606ffd640 1 -- 172.21.15.96:0/379763936 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fb60c00ccc0 con 0x7fb61009b360 2024-03-21T06:39:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.673+0000 7fb61860f640 1 -- 172.21.15.96:0/379763936 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb610062f10 con 0x7fb61009b360 2024-03-21T06:39:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.673+0000 7fb606ffd640 1 --2- 172.21.15.96:0/379763936 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb5f4038260 0x7fb5f403a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.673+0000 7fb606ffd640 1 -- 172.21.15.96:0/379763936 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(713..713 src has 402..713) v4 ==== 3403+0+0 (crc 0 0 0) 0x7fb60c0480b0 con 0x7fb61009b360 2024-03-21T06:39:02.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.673+0000 7fb615baa640 1 --2- 172.21.15.96:0/379763936 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb5f4038260 0x7fb5f403a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:02.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.674+0000 7fb615baa640 1 --2- 172.21.15.96:0/379763936 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb5f4038260 0x7fb5f403a770 crc :-1 s=READY pgs=3867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:02.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.678+0000 7fb606ffd640 1 -- 172.21.15.96:0/379763936 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fb60c00a7b0 con 0x7fb61009b360 2024-03-21T06:39:02.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.857+0000 7fb61860f640 1 -- 172.21.15.96:0/379763936 --> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fb61009ba80 con 0x7fb5f4038260 2024-03-21T06:39:02.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.858+0000 7fb606ffd640 1 -- 172.21.15.96:0/379763936 <== mgr.4099 v2:172.21.15.184:6800/2376803682 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (crc 0 0 0) 0x7fb6000124e0 con 0x7fb5f4038260 2024-03-21T06:39:02.857 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 0 to deep-scrub 2024-03-21T06:39:02.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.861+0000 7fb61860f640 1 -- 172.21.15.96:0/379763936 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb5f4038260 msgr2=0x7fb5f403a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:02.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.861+0000 7fb61860f640 1 --2- 172.21.15.96:0/379763936 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb5f4038260 0x7fb5f403a770 crc :-1 s=READY pgs=3867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:02.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.861+0000 7fb61860f640 1 -- 172.21.15.96:0/379763936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb61009b360 msgr2=0x7fb6101387e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:02.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.861+0000 7fb61860f640 1 --2- 172.21.15.96:0/379763936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb61009b360 0x7fb6101387e0 crc :-1 s=READY pgs=12996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:02.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.861+0000 7fb61860f640 1 -- 172.21.15.96:0/379763936 shutdown_connections 2024-03-21T06:39:02.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.861+0000 7fb61860f640 1 --2- 172.21.15.96:0/379763936 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb5f4038260 0x7fb5f403a770 unknown :-1 s=CLOSED pgs=3867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:02.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.861+0000 7fb61860f640 1 --2- 172.21.15.96:0/379763936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb61009b360 0x7fb6101387e0 unknown :-1 s=CLOSED pgs=12996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:02.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.861+0000 7fb61860f640 1 -- 172.21.15.96:0/379763936 >> 172.21.15.96:0/379763936 conn(0x7fb6100a42c0 msgr2=0x7fb6100a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:02.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.861+0000 7fb61860f640 1 -- 172.21.15.96:0/379763936 shutdown_connections 2024-03-21T06:39:02.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.861+0000 7fb61860f640 1 -- 172.21.15.96:0/379763936 wait complete. 2024-03-21T06:39:02.877 INFO:tasks.ceph:Scrubbing osd.1 2024-03-21T06:39:02.877 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-21T06:39:02.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.981+0000 7f07f691c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:02.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.990+0000 7f07f691c640 1 Processor -- start 2024-03-21T06:39:02.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.990+0000 7f07f691c640 1 -- start start 2024-03-21T06:39:02.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.991+0000 7f07f691c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f07f0098cd0 0x7f07f0136540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:02.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.991+0000 7f07f691c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f009c410 con 0x7f07f00981b0 2024-03-21T06:39:02.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.991+0000 7f07f691c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f009b9b0 con 0x7f07f0098cd0 2024-03-21T06:39:02.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.991+0000 7f07ef7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f07f0098cd0 0x7f07f0136540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:02.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.991+0000 7f07effff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f07f00981b0 0x7f07f00985c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:53162/0 (socket says 172.21.15.96:53162) 2024-03-21T06:39:02.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.991+0000 7f07effff640 1 -- 172.21.15.96:0/1607344272 learned_addr learned my addr 172.21.15.96:0/1607344272 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:39:02.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.991+0000 7f07ef7fe640 1 -- 172.21.15.96:0/1607344272 >> v1:172.21.15.96:6789/0 conn(0x7f07f00981b0 legacy=0x7f07f00985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:02.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.991+0000 7f07ef7fe640 1 -- 172.21.15.96:0/1607344272 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f009bdf0 con 0x7f07f0098cd0 2024-03-21T06:39:02.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.991+0000 7f07ef7fe640 1 --2- 172.21.15.96:0/1607344272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07f0098cd0 0x7f07f0136540 crc :-1 s=READY pgs=12998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbb8a3fe3f681c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:02.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.991+0000 7f07eeffd640 1 -- 172.21.15.96:0/1607344272 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f07e4009ce0 con 0x7f07f0098cd0 2024-03-21T06:39:02.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.991+0000 7f07eeffd640 1 -- 172.21.15.96:0/1607344272 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f07e4009e40 con 0x7f07f0098cd0 2024-03-21T06:39:02.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.991+0000 7f07eeffd640 1 -- 172.21.15.96:0/1607344272 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f07e400e360 con 0x7f07f0098cd0 2024-03-21T06:39:02.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.992+0000 7f07f691c640 1 -- 172.21.15.96:0/1607344272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07f0098cd0 msgr2=0x7f07f0136540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:02.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.992+0000 7f07f691c640 1 --2- 172.21.15.96:0/1607344272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07f0098cd0 0x7f07f0136540 crc :-1 s=READY pgs=12998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:02.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.992+0000 7f07f691c640 1 -- 172.21.15.96:0/1607344272 shutdown_connections 2024-03-21T06:39:02.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.992+0000 7f07f691c640 1 --2- 172.21.15.96:0/1607344272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07f0098cd0 0x7f07f0136540 unknown :-1 s=CLOSED pgs=12998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:02.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.992+0000 7f07f691c640 1 -- 172.21.15.96:0/1607344272 >> 172.21.15.96:0/1607344272 conn(0x7f07f009f2a0 msgr2=0x7f07f009c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:02.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.992+0000 7f07f691c640 1 -- 172.21.15.96:0/1607344272 shutdown_connections 2024-03-21T06:39:02.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.992+0000 7f07f691c640 1 -- 172.21.15.96:0/1607344272 wait complete. 2024-03-21T06:39:02.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.992+0000 7f07f691c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:02.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.993+0000 7f07f691c640 1 Processor -- start 2024-03-21T06:39:02.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.993+0000 7f07f691c640 1 -- start start 2024-03-21T06:39:02.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.994+0000 7f07f691c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07f00981b0 0x7f07f0130190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:02.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.994+0000 7f07f691c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f0131cd0 con 0x7f07f00981b0 2024-03-21T06:39:02.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.994+0000 7f07effff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07f00981b0 0x7f07f0130190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:02.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.994+0000 7f07effff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07f00981b0 0x7f07f0130190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55992/0 (socket says 172.21.15.96:55992) 2024-03-21T06:39:02.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.994+0000 7f07effff640 1 -- 172.21.15.96:0/4090898326 learned_addr learned my addr 172.21.15.96:0/4090898326 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:39:02.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.994+0000 7f07effff640 1 -- 172.21.15.96:0/4090898326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f0131ec0 con 0x7f07f00981b0 2024-03-21T06:39:02.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.994+0000 7f07effff640 1 --2- 172.21.15.96:0/4090898326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07f00981b0 0x7f07f0130190 crc :-1 s=READY pgs=12999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:02.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.995+0000 7f07ecff9640 1 -- 172.21.15.96:0/4090898326 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f07e00058a0 con 0x7f07f00981b0 2024-03-21T06:39:02.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.995+0000 7f07ecff9640 1 -- 172.21.15.96:0/4090898326 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f07e0005a00 con 0x7f07f00981b0 2024-03-21T06:39:02.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.995+0000 7f07f691c640 1 -- 172.21.15.96:0/4090898326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f012da00 con 0x7f07f00981b0 2024-03-21T06:39:02.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.995+0000 7f07ecff9640 1 -- 172.21.15.96:0/4090898326 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f07e0005ca0 con 0x7f07f00981b0 2024-03-21T06:39:02.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.995+0000 7f07f691c640 1 -- 172.21.15.96:0/4090898326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f012e2e0 con 0x7f07f00981b0 2024-03-21T06:39:02.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.995+0000 7f07f691c640 1 -- 172.21.15.96:0/4090898326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f07f006da70 con 0x7f07f00981b0 2024-03-21T06:39:02.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.996+0000 7f07ecff9640 1 -- 172.21.15.96:0/4090898326 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f07e00070b0 con 0x7f07f00981b0 2024-03-21T06:39:02.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.996+0000 7f07ecff9640 1 --2- 172.21.15.96:0/4090898326 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f07dc038210 0x7f07dc03a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:02.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.996+0000 7f07ecff9640 1 -- 172.21.15.96:0/4090898326 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(713..713 src has 402..713) v4 ==== 3403+0+0 (crc 0 0 0) 0x7f07e0047b30 con 0x7f07f00981b0 2024-03-21T06:39:02.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.996+0000 7f07ecff9640 1 --2- 172.21.15.96:0/4090898326 >> v2:172.21.15.96:6805/13988474 conn(0x7f07dc03e330 0x7f07dc040820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:02.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.996+0000 7f07ecff9640 1 -- 172.21.15.96:0/4090898326 --> v2:172.21.15.96:6805/13988474 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07dc040eb0 con 0x7f07dc03e330 2024-03-21T06:39:02.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.996+0000 7f07ecff9640 1 -- 172.21.15.96:0/4090898326 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=713) v2 ==== 24+0+0 (crc 0 0 0) 0x7f07e0047fc0 con 0x7f07f00981b0 2024-03-21T06:39:02.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.996+0000 7f07ef7fe640 1 --2- 172.21.15.96:0/4090898326 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f07dc038210 0x7f07dc03a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:02.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.996+0000 7f07f4eb9640 1 --2- 172.21.15.96:0/4090898326 >> v2:172.21.15.96:6805/13988474 conn(0x7f07dc03e330 0x7f07dc040820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:39:02.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.997+0000 7f07ef7fe640 1 --2- 172.21.15.96:0/4090898326 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f07dc038210 0x7f07dc03a720 crc :-1 s=READY pgs=3868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:02.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.998+0000 7f07f4eb9640 1 --2- 172.21.15.96:0/4090898326 >> v2:172.21.15.96:6805/13988474 conn(0x7f07dc03e330 0x7f07dc040820 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:02.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:02.999+0000 7f07ecff9640 1 -- 172.21.15.96:0/4090898326 <== osd.1 v2:172.21.15.96:6805/13988474 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7f07dc040eb0 con 0x7f07dc03e330 2024-03-21T06:39:03.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.002+0000 7f07f691c640 1 -- 172.21.15.96:0/4090898326 --> v2:172.21.15.96:6805/13988474 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f07f006e5a0 con 0x7f07dc03e330 2024-03-21T06:39:03.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.003+0000 7f07ecff9640 1 -- 172.21.15.96:0/4090898326 <== osd.1 v2:172.21.15.96:6805/13988474 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+22 (crc 0 0 0) 0x7f07e800a820 con 0x7f07dc03e330 2024-03-21T06:39:03.002 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-21T06:39:03.002 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "" 2024-03-21T06:39:03.002 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-21T06:39:03.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.003+0000 7f07f691c640 1 -- 172.21.15.96:0/4090898326 >> v2:172.21.15.96:6805/13988474 conn(0x7f07dc03e330 msgr2=0x7f07dc040820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.003+0000 7f07f691c640 1 --2- 172.21.15.96:0/4090898326 >> v2:172.21.15.96:6805/13988474 conn(0x7f07dc03e330 0x7f07dc040820 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.003+0000 7f07f691c640 1 -- 172.21.15.96:0/4090898326 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f07dc038210 msgr2=0x7f07dc03a720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.003+0000 7f07f691c640 1 --2- 172.21.15.96:0/4090898326 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f07dc038210 0x7f07dc03a720 crc :-1 s=READY pgs=3868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.003+0000 7f07f691c640 1 -- 172.21.15.96:0/4090898326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07f00981b0 msgr2=0x7f07f0130190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.003+0000 7f07f691c640 1 --2- 172.21.15.96:0/4090898326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07f00981b0 0x7f07f0130190 crc :-1 s=READY pgs=12999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.003+0000 7f07f691c640 1 -- 172.21.15.96:0/4090898326 shutdown_connections 2024-03-21T06:39:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.003+0000 7f07f691c640 1 --2- 172.21.15.96:0/4090898326 >> v2:172.21.15.96:6805/13988474 conn(0x7f07dc03e330 0x7f07dc040820 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.003+0000 7f07f691c640 1 --2- 172.21.15.96:0/4090898326 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f07dc038210 0x7f07dc03a720 unknown :-1 s=CLOSED pgs=3868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.003+0000 7f07f691c640 1 --2- 172.21.15.96:0/4090898326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07f00981b0 0x7f07f0130190 unknown :-1 s=CLOSED pgs=12999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.003+0000 7f07f691c640 1 -- 172.21.15.96:0/4090898326 >> 172.21.15.96:0/4090898326 conn(0x7f07f009f2a0 msgr2=0x7f07f0070820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.004+0000 7f07f691c640 1 -- 172.21.15.96:0/4090898326 shutdown_connections 2024-03-21T06:39:03.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.004+0000 7f07f691c640 1 -- 172.21.15.96:0/4090898326 wait complete. 2024-03-21T06:39:03.013 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-21T06:39:03.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.116+0000 7fd1d6d9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:03.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.121+0000 7fd1d6d9c640 1 Processor -- start 2024-03-21T06:39:03.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.121+0000 7fd1d6d9c640 1 -- start start 2024-03-21T06:39:03.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.122+0000 7fd1d6d9c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd1d009b360 0x7fd1d009cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:03.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.122+0000 7fd1d6d9c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d006e410 con 0x7fd1d01315d0 2024-03-21T06:39:03.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.122+0000 7fd1d6d9c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d009b770 con 0x7fd1d009b360 2024-03-21T06:39:03.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.122+0000 7fd1d4b38640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd1d009b360 0x7fd1d009cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:03.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.122+0000 7fd1cffff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fd1d01315d0 0x7fd1d0134130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:53166/0 (socket says 172.21.15.96:53166) 2024-03-21T06:39:03.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.122+0000 7fd1cffff640 1 -- 172.21.15.96:0/2128717875 learned_addr learned my addr 172.21.15.96:0/2128717875 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:39:03.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.123+0000 7fd1d4b38640 1 -- 172.21.15.96:0/2128717875 >> v1:172.21.15.96:6789/0 conn(0x7fd1d01315d0 legacy=0x7fd1d0134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.123+0000 7fd1d4b38640 1 -- 172.21.15.96:0/2128717875 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d006dd00 con 0x7fd1d009b360 2024-03-21T06:39:03.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.123+0000 7fd1d4b38640 1 --2- 172.21.15.96:0/2128717875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d009b360 0x7fd1d009cda0 crc :-1 s=READY pgs=13001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8398c340b614c4ac server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:03.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.123+0000 7fd1cf7fe640 1 -- 172.21.15.96:0/2128717875 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd1b8009ce0 con 0x7fd1d009b360 2024-03-21T06:39:03.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.123+0000 7fd1cf7fe640 1 -- 172.21.15.96:0/2128717875 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd1b8009e40 con 0x7fd1d009b360 2024-03-21T06:39:03.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.123+0000 7fd1cf7fe640 1 -- 172.21.15.96:0/2128717875 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd1b800e360 con 0x7fd1d009b360 2024-03-21T06:39:03.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.123+0000 7fd1d6d9c640 1 -- 172.21.15.96:0/2128717875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d009b360 msgr2=0x7fd1d009cda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.123+0000 7fd1d6d9c640 1 --2- 172.21.15.96:0/2128717875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d009b360 0x7fd1d009cda0 crc :-1 s=READY pgs=13001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.123+0000 7fd1d6d9c640 1 -- 172.21.15.96:0/2128717875 shutdown_connections 2024-03-21T06:39:03.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.123+0000 7fd1d6d9c640 1 --2- 172.21.15.96:0/2128717875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d009b360 0x7fd1d009cda0 unknown :-1 s=CLOSED pgs=13001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.123+0000 7fd1d6d9c640 1 -- 172.21.15.96:0/2128717875 >> 172.21.15.96:0/2128717875 conn(0x7fd1d00a42c0 msgr2=0x7fd1d00a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:03.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.124+0000 7fd1d6d9c640 1 -- 172.21.15.96:0/2128717875 shutdown_connections 2024-03-21T06:39:03.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.124+0000 7fd1d6d9c640 1 -- 172.21.15.96:0/2128717875 wait complete. 2024-03-21T06:39:03.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.124+0000 7fd1d6d9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:03.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.125+0000 7fd1d6d9c640 1 Processor -- start 2024-03-21T06:39:03.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.125+0000 7fd1d6d9c640 1 -- start start 2024-03-21T06:39:03.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.125+0000 7fd1d6d9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d009b360 0x7fd1d011ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:03.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.125+0000 7fd1d6d9c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d0120740 con 0x7fd1d009b360 2024-03-21T06:39:03.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.125+0000 7fd1d4b38640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d009b360 0x7fd1d011ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:03.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.126+0000 7fd1d4b38640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d009b360 0x7fd1d011ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56016/0 (socket says 172.21.15.96:56016) 2024-03-21T06:39:03.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.126+0000 7fd1d4b38640 1 -- 172.21.15.96:0/2723158601 learned_addr learned my addr 172.21.15.96:0/2723158601 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:39:03.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.126+0000 7fd1d4b38640 1 -- 172.21.15.96:0/2723158601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d0130760 con 0x7fd1d009b360 2024-03-21T06:39:03.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.126+0000 7fd1d4b38640 1 --2- 172.21.15.96:0/2723158601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d009b360 0x7fd1d011ca10 crc :-1 s=READY pgs=13002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:03.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.126+0000 7fd1cd7fa640 1 -- 172.21.15.96:0/2723158601 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd1b8014620 con 0x7fd1d009b360 2024-03-21T06:39:03.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.126+0000 7fd1cd7fa640 1 -- 172.21.15.96:0/2723158601 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fd1b8014780 con 0x7fd1d009b360 2024-03-21T06:39:03.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.126+0000 7fd1cd7fa640 1 -- 172.21.15.96:0/2723158601 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fd1b8014a20 con 0x7fd1d009b360 2024-03-21T06:39:03.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.126+0000 7fd1d6d9c640 1 -- 172.21.15.96:0/2723158601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d011eb10 con 0x7fd1d009b360 2024-03-21T06:39:03.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.126+0000 7fd1d6d9c640 1 -- 172.21.15.96:0/2723158601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d011fc40 con 0x7fd1d009b360 2024-03-21T06:39:03.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.127+0000 7fd1cd7fa640 1 -- 172.21.15.96:0/2723158601 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fd1b8014b80 con 0x7fd1d009b360 2024-03-21T06:39:03.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.127+0000 7fd1d6d9c640 1 -- 172.21.15.96:0/2723158601 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d009ba50 con 0x7fd1d009b360 2024-03-21T06:39:03.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.127+0000 7fd1cd7fa640 1 --2- 172.21.15.96:0/2723158601 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd1b0038260 0x7fd1b003a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:03.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.127+0000 7fd1cd7fa640 1 -- 172.21.15.96:0/2723158601 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(713..713 src has 402..713) v4 ==== 3403+0+0 (crc 0 0 0) 0x7fd1b8019040 con 0x7fd1d009b360 2024-03-21T06:39:03.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.128+0000 7fd1cffff640 1 --2- 172.21.15.96:0/2723158601 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd1b0038260 0x7fd1b003a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:03.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.128+0000 7fd1cffff640 1 --2- 172.21.15.96:0/2723158601 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd1b0038260 0x7fd1b003a770 crc :-1 s=READY pgs=3869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:03.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.132+0000 7fd1cd7fa640 1 -- 172.21.15.96:0/2723158601 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fd1b8018c00 con 0x7fd1d009b360 2024-03-21T06:39:03.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.304+0000 7fd1d6d9c640 1 -- 172.21.15.96:0/2723158601 --> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1d011eea0 con 0x7fd1b0038260 2024-03-21T06:39:03.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.304+0000 7fd1cd7fa640 1 -- 172.21.15.96:0/2723158601 <== mgr.4099 v2:172.21.15.184:6800/2376803682 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (crc 0 0 0) 0x7fd1d011eea0 con 0x7fd1b0038260 2024-03-21T06:39:03.304 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 1 to deep-scrub 2024-03-21T06:39:03.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.307+0000 7fd1d6d9c640 1 -- 172.21.15.96:0/2723158601 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd1b0038260 msgr2=0x7fd1b003a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.307+0000 7fd1d6d9c640 1 --2- 172.21.15.96:0/2723158601 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd1b0038260 0x7fd1b003a770 crc :-1 s=READY pgs=3869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.307+0000 7fd1d6d9c640 1 -- 172.21.15.96:0/2723158601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d009b360 msgr2=0x7fd1d011ca10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.307+0000 7fd1d6d9c640 1 --2- 172.21.15.96:0/2723158601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d009b360 0x7fd1d011ca10 crc :-1 s=READY pgs=13002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.307+0000 7fd1d6d9c640 1 -- 172.21.15.96:0/2723158601 shutdown_connections 2024-03-21T06:39:03.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.307+0000 7fd1d6d9c640 1 --2- 172.21.15.96:0/2723158601 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fd1b0038260 0x7fd1b003a770 unknown :-1 s=CLOSED pgs=3869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.307+0000 7fd1d6d9c640 1 --2- 172.21.15.96:0/2723158601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d009b360 0x7fd1d011ca10 unknown :-1 s=CLOSED pgs=13002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.307+0000 7fd1d6d9c640 1 -- 172.21.15.96:0/2723158601 >> 172.21.15.96:0/2723158601 conn(0x7fd1d00a42c0 msgr2=0x7fd1d0133f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:03.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.307+0000 7fd1d6d9c640 1 -- 172.21.15.96:0/2723158601 shutdown_connections 2024-03-21T06:39:03.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.307+0000 7fd1d6d9c640 1 -- 172.21.15.96:0/2723158601 wait complete. 2024-03-21T06:39:03.322 INFO:tasks.ceph:Scrubbing osd.2 2024-03-21T06:39:03.322 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-21T06:39:03.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.425+0000 7f303999c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:03.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.433+0000 7f303999c640 1 Processor -- start 2024-03-21T06:39:03.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.433+0000 7f303999c640 1 -- start start 2024-03-21T06:39:03.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.434+0000 7f303999c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f303409b360 0x7f303409cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:03.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.434+0000 7f303999c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f303406e410 con 0x7f30341315d0 2024-03-21T06:39:03.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.434+0000 7f303999c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f303409b770 con 0x7f303409b360 2024-03-21T06:39:03.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.434+0000 7f3032ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f303409b360 0x7f303409cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:03.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.434+0000 7f30327fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f30341315d0 0x7f3034134130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:53170/0 (socket says 172.21.15.96:53170) 2024-03-21T06:39:03.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.434+0000 7f3032ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f303409b360 0x7f303409cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56024/0 (socket says 172.21.15.96:56024) 2024-03-21T06:39:03.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.434+0000 7f30327fc640 1 -- 172.21.15.96:0/423189389 learned_addr learned my addr 172.21.15.96:0/423189389 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:39:03.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.434+0000 7f3032ffd640 1 -- 172.21.15.96:0/423189389 >> v1:172.21.15.96:6789/0 conn(0x7f30341315d0 legacy=0x7f3034134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.434+0000 7f3032ffd640 1 -- 172.21.15.96:0/423189389 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303406dd00 con 0x7f303409b360 2024-03-21T06:39:03.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.435+0000 7f3032ffd640 1 --2- 172.21.15.96:0/423189389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f303409b360 0x7f303409cda0 crc :-1 s=READY pgs=13004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d80f67f457062c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:03.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.435+0000 7f3031ffb640 1 -- 172.21.15.96:0/423189389 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f302c00e040 con 0x7f303409b360 2024-03-21T06:39:03.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.435+0000 7f3031ffb640 1 -- 172.21.15.96:0/423189389 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f302c007670 con 0x7f303409b360 2024-03-21T06:39:03.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.435+0000 7f3031ffb640 1 -- 172.21.15.96:0/423189389 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f302c007910 con 0x7f303409b360 2024-03-21T06:39:03.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.435+0000 7f303999c640 1 -- 172.21.15.96:0/423189389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f303409b360 msgr2=0x7f303409cda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.435+0000 7f303999c640 1 --2- 172.21.15.96:0/423189389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f303409b360 0x7f303409cda0 crc :-1 s=READY pgs=13004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.435+0000 7f303999c640 1 -- 172.21.15.96:0/423189389 shutdown_connections 2024-03-21T06:39:03.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.435+0000 7f303999c640 1 --2- 172.21.15.96:0/423189389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f303409b360 0x7f303409cda0 unknown :-1 s=CLOSED pgs=13004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.435+0000 7f303999c640 1 -- 172.21.15.96:0/423189389 >> 172.21.15.96:0/423189389 conn(0x7f30340a42c0 msgr2=0x7f30340a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:03.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.435+0000 7f303999c640 1 -- 172.21.15.96:0/423189389 shutdown_connections 2024-03-21T06:39:03.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.435+0000 7f303999c640 1 -- 172.21.15.96:0/423189389 wait complete. 2024-03-21T06:39:03.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.436+0000 7f303999c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:03.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.436+0000 7f303999c640 1 Processor -- start 2024-03-21T06:39:03.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.437+0000 7f303999c640 1 -- start start 2024-03-21T06:39:03.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.437+0000 7f303999c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f303409b360 0x7f303411ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:03.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.437+0000 7f303999c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303411ed50 con 0x7f303409b360 2024-03-21T06:39:03.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.437+0000 7f3032ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f303409b360 0x7f303411ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:03.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.437+0000 7f3032ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f303409b360 0x7f303411ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56028/0 (socket says 172.21.15.96:56028) 2024-03-21T06:39:03.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.437+0000 7f3032ffd640 1 -- 172.21.15.96:0/720753490 learned_addr learned my addr 172.21.15.96:0/720753490 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:39:03.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.438+0000 7f3032ffd640 1 -- 172.21.15.96:0/720753490 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303411ef80 con 0x7f303409b360 2024-03-21T06:39:03.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.438+0000 7f3032ffd640 1 --2- 172.21.15.96:0/720753490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f303409b360 0x7f303411ca10 crc :-1 s=READY pgs=13005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:03.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.438+0000 7f302bfff640 1 -- 172.21.15.96:0/720753490 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f302c00e040 con 0x7f303409b360 2024-03-21T06:39:03.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.438+0000 7f302bfff640 1 -- 172.21.15.96:0/720753490 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f302c012a80 con 0x7f303409b360 2024-03-21T06:39:03.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.438+0000 7f303999c640 1 -- 172.21.15.96:0/720753490 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3034120560 con 0x7f303409b360 2024-03-21T06:39:03.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.438+0000 7f302bfff640 1 -- 172.21.15.96:0/720753490 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f302c012d20 con 0x7f303409b360 2024-03-21T06:39:03.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.438+0000 7f303999c640 1 -- 172.21.15.96:0/720753490 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3034120140 con 0x7f303409b360 2024-03-21T06:39:03.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.439+0000 7f302bfff640 1 -- 172.21.15.96:0/720753490 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f302c0153f0 con 0x7f303409b360 2024-03-21T06:39:03.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.439+0000 7f302bfff640 1 --2- 172.21.15.96:0/720753490 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3014038210 0x7f301403a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:03.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.439+0000 7f303999c640 1 -- 172.21.15.96:0/720753490 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f303411f110 con 0x7f303409b360 2024-03-21T06:39:03.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.439+0000 7f302bfff640 1 -- 172.21.15.96:0/720753490 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(713..713 src has 402..713) v4 ==== 3403+0+0 (crc 0 0 0) 0x7f302c049580 con 0x7f303409b360 2024-03-21T06:39:03.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.439+0000 7f302bfff640 1 --2- 172.21.15.96:0/720753490 >> v2:172.21.15.184:6804/155456265 conn(0x7f301403e2d0 0x7f30140407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:03.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.439+0000 7f302bfff640 1 -- 172.21.15.96:0/720753490 --> v2:172.21.15.184:6804/155456265 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3014040e50 con 0x7f301403e2d0 2024-03-21T06:39:03.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.439+0000 7f302bfff640 1 -- 172.21.15.96:0/720753490 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=713) v2 ==== 24+0+0 (crc 0 0 0) 0x7f302c00fe80 con 0x7f303409b360 2024-03-21T06:39:03.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.439+0000 7f30327fc640 1 --2- 172.21.15.96:0/720753490 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3014038210 0x7f301403a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:03.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.440+0000 7f30327fc640 1 --2- 172.21.15.96:0/720753490 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3014038210 0x7f301403a720 crc :-1 s=READY pgs=3870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:03.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.440+0000 7f30337fe640 1 --2- 172.21.15.96:0/720753490 >> v2:172.21.15.184:6804/155456265 conn(0x7f301403e2d0 0x7f30140407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:39:03.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.442+0000 7f30337fe640 1 --2- 172.21.15.96:0/720753490 >> v2:172.21.15.184:6804/155456265 conn(0x7f301403e2d0 0x7f30140407c0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:03.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.443+0000 7f302bfff640 1 -- 172.21.15.96:0/720753490 <== osd.2 v2:172.21.15.184:6804/155456265 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7f3014040e50 con 0x7f301403e2d0 2024-03-21T06:39:03.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.446+0000 7f303999c640 1 -- 172.21.15.96:0/720753490 --> v2:172.21.15.184:6804/155456265 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f303411f9d0 con 0x7f301403e2d0 2024-03-21T06:39:03.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.448+0000 7f302bfff640 1 -- 172.21.15.96:0/720753490 <== osd.2 v2:172.21.15.184:6804/155456265 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+22 (crc 0 0 0) 0x7f303411f9d0 con 0x7f301403e2d0 2024-03-21T06:39:03.447 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-21T06:39:03.447 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "" 2024-03-21T06:39:03.447 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-21T06:39:03.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.448+0000 7f303999c640 1 -- 172.21.15.96:0/720753490 >> v2:172.21.15.184:6804/155456265 conn(0x7f301403e2d0 msgr2=0x7f30140407c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.448+0000 7f303999c640 1 --2- 172.21.15.96:0/720753490 >> v2:172.21.15.184:6804/155456265 conn(0x7f301403e2d0 0x7f30140407c0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.448+0000 7f303999c640 1 -- 172.21.15.96:0/720753490 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3014038210 msgr2=0x7f301403a720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.448+0000 7f303999c640 1 --2- 172.21.15.96:0/720753490 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3014038210 0x7f301403a720 crc :-1 s=READY pgs=3870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.448+0000 7f303999c640 1 -- 172.21.15.96:0/720753490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f303409b360 msgr2=0x7f303411ca10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.448+0000 7f303999c640 1 --2- 172.21.15.96:0/720753490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f303409b360 0x7f303411ca10 crc :-1 s=READY pgs=13005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.448+0000 7f303999c640 1 -- 172.21.15.96:0/720753490 shutdown_connections 2024-03-21T06:39:03.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.448+0000 7f303999c640 1 --2- 172.21.15.96:0/720753490 >> v2:172.21.15.184:6804/155456265 conn(0x7f301403e2d0 0x7f30140407c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.448+0000 7f303999c640 1 --2- 172.21.15.96:0/720753490 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3014038210 0x7f301403a720 unknown :-1 s=CLOSED pgs=3870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.448+0000 7f303999c640 1 --2- 172.21.15.96:0/720753490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f303409b360 0x7f303411ca10 unknown :-1 s=CLOSED pgs=13005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.448+0000 7f303999c640 1 -- 172.21.15.96:0/720753490 >> 172.21.15.96:0/720753490 conn(0x7f30340a42c0 msgr2=0x7f30341bc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:03.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.448+0000 7f303999c640 1 -- 172.21.15.96:0/720753490 shutdown_connections 2024-03-21T06:39:03.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.448+0000 7f303999c640 1 -- 172.21.15.96:0/720753490 wait complete. 2024-03-21T06:39:03.458 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-21T06:39:03.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.561+0000 7fb33c26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:03.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.566+0000 7fb33c26d640 1 Processor -- start 2024-03-21T06:39:03.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.566+0000 7fb33c26d640 1 -- start start 2024-03-21T06:39:03.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.567+0000 7fb33c26d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb33409b5d0 0x7fb334130460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:03.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.567+0000 7fb33c26d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb334099dc0 con 0x7fb3341309a0 2024-03-21T06:39:03.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.567+0000 7fb33c26d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33409bae0 con 0x7fb33409b5d0 2024-03-21T06:39:03.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.567+0000 7fb33a009640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb33409b5d0 0x7fb334130460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:03.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.567+0000 7fb339808640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fb3341309a0 0x7fb334098190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:53184/0 (socket says 172.21.15.96:53184) 2024-03-21T06:39:03.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.567+0000 7fb339808640 1 -- 172.21.15.96:0/2467447459 learned_addr learned my addr 172.21.15.96:0/2467447459 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:39:03.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.567+0000 7fb33a009640 1 -- 172.21.15.96:0/2467447459 >> v1:172.21.15.96:6789/0 conn(0x7fb3341309a0 legacy=0x7fb334098190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.567+0000 7fb33a009640 1 -- 172.21.15.96:0/2467447459 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33409c0f0 con 0x7fb33409b5d0 2024-03-21T06:39:03.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.567+0000 7fb339007640 1 -- 172.21.15.96:0/2467447459 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622436764 0 0) 0x7fb334099dc0 con 0x7fb3341309a0 2024-03-21T06:39:03.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.567+0000 7fb33a009640 1 --2- 172.21.15.96:0/2467447459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb33409b5d0 0x7fb334130460 crc :-1 s=READY pgs=13007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488b060d1c49b7c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:03.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.567+0000 7fb339007640 1 -- 172.21.15.96:0/2467447459 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb330009ce0 con 0x7fb33409b5d0 2024-03-21T06:39:03.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.567+0000 7fb339007640 1 -- 172.21.15.96:0/2467447459 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb330009e40 con 0x7fb33409b5d0 2024-03-21T06:39:03.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.567+0000 7fb339007640 1 -- 172.21.15.96:0/2467447459 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb33000e360 con 0x7fb33409b5d0 2024-03-21T06:39:03.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.568+0000 7fb33c26d640 1 -- 172.21.15.96:0/2467447459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb33409b5d0 msgr2=0x7fb334130460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.568+0000 7fb33c26d640 1 --2- 172.21.15.96:0/2467447459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb33409b5d0 0x7fb334130460 crc :-1 s=READY pgs=13007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.568+0000 7fb33c26d640 1 -- 172.21.15.96:0/2467447459 shutdown_connections 2024-03-21T06:39:03.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.568+0000 7fb33c26d640 1 --2- 172.21.15.96:0/2467447459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb33409b5d0 0x7fb334130460 unknown :-1 s=CLOSED pgs=13007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.568+0000 7fb33c26d640 1 -- 172.21.15.96:0/2467447459 >> 172.21.15.96:0/2467447459 conn(0x7fb33409fdb0 msgr2=0x7fb3340a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:03.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.568+0000 7fb33c26d640 1 -- 172.21.15.96:0/2467447459 shutdown_connections 2024-03-21T06:39:03.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.568+0000 7fb33c26d640 1 -- 172.21.15.96:0/2467447459 wait complete. 2024-03-21T06:39:03.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.568+0000 7fb33c26d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:03.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.569+0000 7fb33c26d640 1 Processor -- start 2024-03-21T06:39:03.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.569+0000 7fb33c26d640 1 -- start start 2024-03-21T06:39:03.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.570+0000 7fb33c26d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb33409b5d0 0x7fb3341c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:03.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.570+0000 7fb33c26d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3341c8bf0 con 0x7fb33409b5d0 2024-03-21T06:39:03.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.570+0000 7fb33a009640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb33409b5d0 0x7fb3341c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:03.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.570+0000 7fb33a009640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb33409b5d0 0x7fb3341c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56036/0 (socket says 172.21.15.96:56036) 2024-03-21T06:39:03.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.570+0000 7fb33a009640 1 -- 172.21.15.96:0/2603111955 learned_addr learned my addr 172.21.15.96:0/2603111955 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:39:03.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.570+0000 7fb33a009640 1 -- 172.21.15.96:0/2603111955 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb334137d70 con 0x7fb33409b5d0 2024-03-21T06:39:03.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.570+0000 7fb33a009640 1 --2- 172.21.15.96:0/2603111955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb33409b5d0 0x7fb3341c8fb0 crc :-1 s=READY pgs=13008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:03.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.570+0000 7fb326ffd640 1 -- 172.21.15.96:0/2603111955 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb330014620 con 0x7fb33409b5d0 2024-03-21T06:39:03.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.570+0000 7fb326ffd640 1 -- 172.21.15.96:0/2603111955 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fb330014780 con 0x7fb33409b5d0 2024-03-21T06:39:03.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.570+0000 7fb326ffd640 1 -- 172.21.15.96:0/2603111955 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fb330014a20 con 0x7fb33409b5d0 2024-03-21T06:39:03.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.570+0000 7fb33c26d640 1 -- 172.21.15.96:0/2603111955 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3341366a0 con 0x7fb33409b5d0 2024-03-21T06:39:03.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.571+0000 7fb33c26d640 1 -- 172.21.15.96:0/2603111955 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb334072d30 con 0x7fb33409b5d0 2024-03-21T06:39:03.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.571+0000 7fb326ffd640 1 -- 172.21.15.96:0/2603111955 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fb330014bd0 con 0x7fb33409b5d0 2024-03-21T06:39:03.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.571+0000 7fb33c26d640 1 -- 172.21.15.96:0/2603111955 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb334062f10 con 0x7fb33409b5d0 2024-03-21T06:39:03.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.572+0000 7fb326ffd640 1 --2- 172.21.15.96:0/2603111955 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb318038260 0x7fb31803a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:03.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.572+0000 7fb326ffd640 1 -- 172.21.15.96:0/2603111955 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(713..713 src has 402..713) v4 ==== 3403+0+0 (crc 0 0 0) 0x7fb330019040 con 0x7fb33409b5d0 2024-03-21T06:39:03.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.572+0000 7fb339808640 1 --2- 172.21.15.96:0/2603111955 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb318038260 0x7fb31803a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:03.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.572+0000 7fb339808640 1 --2- 172.21.15.96:0/2603111955 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb318038260 0x7fb31803a770 crc :-1 s=READY pgs=3871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:03.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.577+0000 7fb326ffd640 1 -- 172.21.15.96:0/2603111955 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fb330018190 con 0x7fb33409b5d0 2024-03-21T06:39:03.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.751+0000 7fb33c26d640 1 -- 172.21.15.96:0/2603111955 --> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb33409e060 con 0x7fb318038260 2024-03-21T06:39:03.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.751+0000 7fb326ffd640 1 -- 172.21.15.96:0/2603111955 <== mgr.4099 v2:172.21.15.184:6800/2376803682 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (crc 0 0 0) 0x7fb328001250 con 0x7fb318038260 2024-03-21T06:39:03.751 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 2 to deep-scrub 2024-03-21T06:39:03.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.753+0000 7fb33c26d640 1 -- 172.21.15.96:0/2603111955 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb318038260 msgr2=0x7fb31803a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.753+0000 7fb33c26d640 1 --2- 172.21.15.96:0/2603111955 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb318038260 0x7fb31803a770 crc :-1 s=READY pgs=3871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.753+0000 7fb33c26d640 1 -- 172.21.15.96:0/2603111955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb33409b5d0 msgr2=0x7fb3341c8fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.753+0000 7fb33c26d640 1 --2- 172.21.15.96:0/2603111955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb33409b5d0 0x7fb3341c8fb0 crc :-1 s=READY pgs=13008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.753+0000 7fb33c26d640 1 -- 172.21.15.96:0/2603111955 shutdown_connections 2024-03-21T06:39:03.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.753+0000 7fb33c26d640 1 --2- 172.21.15.96:0/2603111955 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fb318038260 0x7fb31803a770 unknown :-1 s=CLOSED pgs=3871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.753+0000 7fb33c26d640 1 --2- 172.21.15.96:0/2603111955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb33409b5d0 0x7fb3341c8fb0 unknown :-1 s=CLOSED pgs=13008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.753+0000 7fb33c26d640 1 -- 172.21.15.96:0/2603111955 >> 172.21.15.96:0/2603111955 conn(0x7fb33409fdb0 msgr2=0x7fb3340a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:03.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.754+0000 7fb33c26d640 1 -- 172.21.15.96:0/2603111955 shutdown_connections 2024-03-21T06:39:03.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.754+0000 7fb33c26d640 1 -- 172.21.15.96:0/2603111955 wait complete. 2024-03-21T06:39:03.765 INFO:tasks.ceph:Scrubbing osd.3 2024-03-21T06:39:03.766 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-21T06:39:03.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.873+0000 7f70108ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:03.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.881+0000 7f70108ad640 1 Processor -- start 2024-03-21T06:39:03.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.881+0000 7f70108ad640 1 -- start start 2024-03-21T06:39:03.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.882+0000 7f70108ad640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f700c09a330 0x7f700c130c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:03.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.882+0000 7f70108ad640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700c09b380 con 0x7f700c099f20 2024-03-21T06:39:03.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.882+0000 7f70108ad640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700c099490 con 0x7f700c09a330 2024-03-21T06:39:03.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.882+0000 7f7009d9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f700c09a330 0x7f700c130c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:03.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.882+0000 7f700a59c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f700c099f20 0x7f700c130540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:53190/0 (socket says 172.21.15.96:53190) 2024-03-21T06:39:03.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.882+0000 7f700a59c640 1 -- 172.21.15.96:0/1733291316 learned_addr learned my addr 172.21.15.96:0/1733291316 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:39:03.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.882+0000 7f7009d9b640 1 -- 172.21.15.96:0/1733291316 >> v1:172.21.15.96:6789/0 conn(0x7f700c099f20 legacy=0x7f700c130540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.882+0000 7f7009d9b640 1 -- 172.21.15.96:0/1733291316 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c099a40 con 0x7f700c09a330 2024-03-21T06:39:03.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.882+0000 7f7009d9b640 1 --2- 172.21.15.96:0/1733291316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f700c09a330 0x7f700c130c50 crc :-1 s=READY pgs=13010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c716a608238e2b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:03.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.882+0000 7f700959a640 1 -- 172.21.15.96:0/1733291316 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6ff4009ce0 con 0x7f700c09a330 2024-03-21T06:39:03.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.882+0000 7f700959a640 1 -- 172.21.15.96:0/1733291316 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f6ff4009e40 con 0x7f700c09a330 2024-03-21T06:39:03.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.882+0000 7f700959a640 1 -- 172.21.15.96:0/1733291316 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f6ff400e360 con 0x7f700c09a330 2024-03-21T06:39:03.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.883+0000 7f70108ad640 1 -- 172.21.15.96:0/1733291316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f700c09a330 msgr2=0x7f700c130c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.883+0000 7f70108ad640 1 --2- 172.21.15.96:0/1733291316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f700c09a330 0x7f700c130c50 crc :-1 s=READY pgs=13010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.883+0000 7f70108ad640 1 -- 172.21.15.96:0/1733291316 shutdown_connections 2024-03-21T06:39:03.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.883+0000 7f70108ad640 1 --2- 172.21.15.96:0/1733291316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f700c09a330 0x7f700c130c50 unknown :-1 s=CLOSED pgs=13010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.883+0000 7f70108ad640 1 -- 172.21.15.96:0/1733291316 >> 172.21.15.96:0/1733291316 conn(0x7f700c09f2a0 msgr2=0x7f700c09c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:03.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.883+0000 7f70108ad640 1 -- 172.21.15.96:0/1733291316 shutdown_connections 2024-03-21T06:39:03.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.883+0000 7f70108ad640 1 -- 172.21.15.96:0/1733291316 wait complete. 2024-03-21T06:39:03.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.883+0000 7f70108ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:03.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.884+0000 7f70108ad640 1 Processor -- start 2024-03-21T06:39:03.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.885+0000 7f70108ad640 1 -- start start 2024-03-21T06:39:03.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.885+0000 7f70108ad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f700c099f20 0x7f700c070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:03.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.885+0000 7f70108ad640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c072030 con 0x7f700c099f20 2024-03-21T06:39:03.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.885+0000 7f700a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f700c099f20 0x7f700c070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:03.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.885+0000 7f700a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f700c099f20 0x7f700c070ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56056/0 (socket says 172.21.15.96:56056) 2024-03-21T06:39:03.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.885+0000 7f700a59c640 1 -- 172.21.15.96:0/3710808651 learned_addr learned my addr 172.21.15.96:0/3710808651 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:39:03.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.886+0000 7f700a59c640 1 -- 172.21.15.96:0/3710808651 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c12c0c0 con 0x7f700c099f20 2024-03-21T06:39:03.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.886+0000 7f700a59c640 1 --2- 172.21.15.96:0/3710808651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f700c099f20 0x7f700c070ce0 crc :-1 s=READY pgs=13011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:03.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.886+0000 7f7002ffd640 1 -- 172.21.15.96:0/3710808651 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f70040059d0 con 0x7f700c099f20 2024-03-21T06:39:03.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.886+0000 7f7002ffd640 1 -- 172.21.15.96:0/3710808651 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f7004005b30 con 0x7f700c099f20 2024-03-21T06:39:03.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.886+0000 7f70108ad640 1 -- 172.21.15.96:0/3710808651 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c06b6e0 con 0x7f700c099f20 2024-03-21T06:39:03.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.886+0000 7f7002ffd640 1 -- 172.21.15.96:0/3710808651 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f7004005dd0 con 0x7f700c099f20 2024-03-21T06:39:03.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.886+0000 7f70108ad640 1 -- 172.21.15.96:0/3710808651 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c072d30 con 0x7f700c099f20 2024-03-21T06:39:03.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.887+0000 7f7002ffd640 1 -- 172.21.15.96:0/3710808651 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f70040070b0 con 0x7f700c099f20 2024-03-21T06:39:03.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.887+0000 7f7002ffd640 1 --2- 172.21.15.96:0/3710808651 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f6fec038260 0x7f6fec03a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:03.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.887+0000 7f70108ad640 1 -- 172.21.15.96:0/3710808651 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f700c06dcc0 con 0x7f700c099f20 2024-03-21T06:39:03.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.887+0000 7f7002ffd640 1 -- 172.21.15.96:0/3710808651 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(713..713 src has 402..713) v4 ==== 3403+0+0 (crc 0 0 0) 0x7f70040479a0 con 0x7f700c099f20 2024-03-21T06:39:03.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.887+0000 7f7002ffd640 1 --2- 172.21.15.96:0/3710808651 >> v2:172.21.15.184:6811/276480837 conn(0x7f6fec03e380 0x7f6fec040870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:03.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.887+0000 7f7002ffd640 1 -- 172.21.15.96:0/3710808651 --> v2:172.21.15.184:6811/276480837 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6fec040f00 con 0x7f6fec03e380 2024-03-21T06:39:03.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.887+0000 7f7002ffd640 1 -- 172.21.15.96:0/3710808651 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=713) v2 ==== 24+0+0 (crc 0 0 0) 0x7f700400d510 con 0x7f700c099f20 2024-03-21T06:39:03.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.887+0000 7f7009d9b640 1 --2- 172.21.15.96:0/3710808651 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f6fec038260 0x7f6fec03a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:03.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.888+0000 7f700ad9d640 1 --2- 172.21.15.96:0/3710808651 >> v2:172.21.15.184:6811/276480837 conn(0x7f6fec03e380 0x7f6fec040870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-21T06:39:03.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.888+0000 7f7009d9b640 1 --2- 172.21.15.96:0/3710808651 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f6fec038260 0x7f6fec03a770 crc :-1 s=READY pgs=3872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:03.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.890+0000 7f700ad9d640 1 --2- 172.21.15.96:0/3710808651 >> v2:172.21.15.184:6811/276480837 conn(0x7f6fec03e380 0x7f6fec040870 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:03.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.891+0000 7f7002ffd640 1 -- 172.21.15.96:0/3710808651 <== osd.3 v2:172.21.15.184:6811/276480837 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+5172 (crc 0 0 0) 0x7f6fec040f00 con 0x7f6fec03e380 2024-03-21T06:39:03.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.895+0000 7f70108ad640 1 -- 172.21.15.96:0/3710808651 --> v2:172.21.15.184:6811/276480837 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f700c06d900 con 0x7f6fec03e380 2024-03-21T06:39:03.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.896+0000 7f7002ffd640 1 -- 172.21.15.96:0/3710808651 <== osd.3 v2:172.21.15.184:6811/276480837 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+22 (crc 0 0 0) 0x7f6ffc00a820 con 0x7f6fec03e380 2024-03-21T06:39:03.895 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-21T06:39:03.895 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "" 2024-03-21T06:39:03.895 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-21T06:39:03.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.896+0000 7f70108ad640 1 -- 172.21.15.96:0/3710808651 >> v2:172.21.15.184:6811/276480837 conn(0x7f6fec03e380 msgr2=0x7f6fec040870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.896+0000 7f70108ad640 1 --2- 172.21.15.96:0/3710808651 >> v2:172.21.15.184:6811/276480837 conn(0x7f6fec03e380 0x7f6fec040870 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.896+0000 7f70108ad640 1 -- 172.21.15.96:0/3710808651 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f6fec038260 msgr2=0x7f6fec03a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.896+0000 7f70108ad640 1 --2- 172.21.15.96:0/3710808651 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f6fec038260 0x7f6fec03a770 crc :-1 s=READY pgs=3872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.896+0000 7f70108ad640 1 -- 172.21.15.96:0/3710808651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f700c099f20 msgr2=0x7f700c070ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:03.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.896+0000 7f70108ad640 1 --2- 172.21.15.96:0/3710808651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f700c099f20 0x7f700c070ce0 crc :-1 s=READY pgs=13011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.897+0000 7f70108ad640 1 -- 172.21.15.96:0/3710808651 shutdown_connections 2024-03-21T06:39:03.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.897+0000 7f70108ad640 1 --2- 172.21.15.96:0/3710808651 >> v2:172.21.15.184:6811/276480837 conn(0x7f6fec03e380 0x7f6fec040870 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.897+0000 7f70108ad640 1 --2- 172.21.15.96:0/3710808651 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f6fec038260 0x7f6fec03a770 unknown :-1 s=CLOSED pgs=3872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.897+0000 7f70108ad640 1 --2- 172.21.15.96:0/3710808651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f700c099f20 0x7f700c070ce0 unknown :-1 s=CLOSED pgs=13011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:03.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.897+0000 7f70108ad640 1 -- 172.21.15.96:0/3710808651 >> 172.21.15.96:0/3710808651 conn(0x7f700c09f2a0 msgr2=0x7f700c09c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:03.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.897+0000 7f70108ad640 1 -- 172.21.15.96:0/3710808651 shutdown_connections 2024-03-21T06:39:03.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:03.897+0000 7f70108ad640 1 -- 172.21.15.96:0/3710808651 wait complete. 2024-03-21T06:39:03.906 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-21T06:39:04.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.010+0000 7fc3146dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:04.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.015+0000 7fc3146dc640 1 Processor -- start 2024-03-21T06:39:04.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.015+0000 7fc3146dc640 1 -- start start 2024-03-21T06:39:04.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.016+0000 7fc3146dc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc30c09b460 0x7fc30c136460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:04.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.016+0000 7fc3146dc640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc30c099960 con 0x7fc30c06d9e0 2024-03-21T06:39:04.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.016+0000 7fc3146dc640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc30c09bf80 con 0x7fc30c09b460 2024-03-21T06:39:04.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.016+0000 7fc311c77640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc30c09b460 0x7fc30c136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.016+0000 7fc312478640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fc30c06d9e0 0x7fc30c09ad50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:53194/0 (socket says 172.21.15.96:53194) 2024-03-21T06:39:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.016+0000 7fc312478640 1 -- 172.21.15.96:0/3410532436 learned_addr learned my addr 172.21.15.96:0/3410532436 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:39:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.016+0000 7fc311c77640 1 -- 172.21.15.96:0/3410532436 >> v1:172.21.15.96:6789/0 conn(0x7fc30c06d9e0 legacy=0x7fc30c09ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.016+0000 7fc311c77640 1 -- 172.21.15.96:0/3410532436 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30c09c1c0 con 0x7fc30c09b460 2024-03-21T06:39:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.016+0000 7fc311c77640 1 --2- 172.21.15.96:0/3410532436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc30c09b460 0x7fc30c136460 crc :-1 s=READY pgs=13013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5545a8ebcb94f649 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.016+0000 7fc311476640 1 -- 172.21.15.96:0/3410532436 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc2fc009ce0 con 0x7fc30c09b460 2024-03-21T06:39:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.016+0000 7fc311476640 1 -- 172.21.15.96:0/3410532436 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc2fc009e40 con 0x7fc30c09b460 2024-03-21T06:39:04.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.016+0000 7fc311476640 1 -- 172.21.15.96:0/3410532436 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc2fc00e360 con 0x7fc30c09b460 2024-03-21T06:39:04.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.017+0000 7fc3146dc640 1 -- 172.21.15.96:0/3410532436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc30c09b460 msgr2=0x7fc30c136460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:04.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.017+0000 7fc3146dc640 1 --2- 172.21.15.96:0/3410532436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc30c09b460 0x7fc30c136460 crc :-1 s=READY pgs=13013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:04.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.017+0000 7fc3146dc640 1 -- 172.21.15.96:0/3410532436 shutdown_connections 2024-03-21T06:39:04.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.017+0000 7fc3146dc640 1 --2- 172.21.15.96:0/3410532436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc30c09b460 0x7fc30c136460 unknown :-1 s=CLOSED pgs=13013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:04.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.017+0000 7fc3146dc640 1 -- 172.21.15.96:0/3410532436 >> 172.21.15.96:0/3410532436 conn(0x7fc30c09fdb0 msgr2=0x7fc30c0a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:04.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.017+0000 7fc3146dc640 1 -- 172.21.15.96:0/3410532436 shutdown_connections 2024-03-21T06:39:04.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.017+0000 7fc3146dc640 1 -- 172.21.15.96:0/3410532436 wait complete. 2024-03-21T06:39:04.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.017+0000 7fc3146dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:04.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.018+0000 7fc3146dc640 1 Processor -- start 2024-03-21T06:39:04.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.018+0000 7fc3146dc640 1 -- start start 2024-03-21T06:39:04.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.019+0000 7fc3146dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc30c09b460 0x7fc30c1c5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:04.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.019+0000 7fc3146dc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c130650 con 0x7fc30c09b460 2024-03-21T06:39:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.019+0000 7fc312478640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc30c09b460 0x7fc30c1c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.019+0000 7fc312478640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc30c09b460 0x7fc30c1c5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56074/0 (socket says 172.21.15.96:56074) 2024-03-21T06:39:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.019+0000 7fc312478640 1 -- 172.21.15.96:0/2395623168 learned_addr learned my addr 172.21.15.96:0/2395623168 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:39:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.019+0000 7fc312478640 1 -- 172.21.15.96:0/2395623168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30c1303b0 con 0x7fc30c09b460 2024-03-21T06:39:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.019+0000 7fc312478640 1 --2- 172.21.15.96:0/2395623168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc30c09b460 0x7fc30c1c5500 crc :-1 s=READY pgs=13014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.019+0000 7fc302ffd640 1 -- 172.21.15.96:0/2395623168 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc3080059d0 con 0x7fc30c09b460 2024-03-21T06:39:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.019+0000 7fc302ffd640 1 -- 172.21.15.96:0/2395623168 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fc308005b30 con 0x7fc30c09b460 2024-03-21T06:39:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.019+0000 7fc302ffd640 1 -- 172.21.15.96:0/2395623168 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fc308005dd0 con 0x7fc30c09b460 2024-03-21T06:39:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.019+0000 7fc3146dc640 1 -- 172.21.15.96:0/2395623168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30c130d40 con 0x7fc30c09b460 2024-03-21T06:39:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.020+0000 7fc3146dc640 1 -- 172.21.15.96:0/2395623168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30c131de0 con 0x7fc30c09b460 2024-03-21T06:39:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.020+0000 7fc302ffd640 1 -- 172.21.15.96:0/2395623168 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fc3080070b0 con 0x7fc30c09b460 2024-03-21T06:39:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.020+0000 7fc3146dc640 1 -- 172.21.15.96:0/2395623168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30c062f10 con 0x7fc30c09b460 2024-03-21T06:39:04.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.021+0000 7fc302ffd640 1 --2- 172.21.15.96:0/2395623168 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc2f0038260 0x7fc2f003a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:04.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.021+0000 7fc302ffd640 1 -- 172.21.15.96:0/2395623168 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(713..713 src has 402..713) v4 ==== 3403+0+0 (crc 0 0 0) 0x7fc308047b20 con 0x7fc30c09b460 2024-03-21T06:39:04.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.021+0000 7fc311c77640 1 --2- 172.21.15.96:0/2395623168 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc2f0038260 0x7fc2f003a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:04.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.021+0000 7fc311c77640 1 --2- 172.21.15.96:0/2395623168 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc2f0038260 0x7fc2f003a770 crc :-1 s=READY pgs=3873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:04.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.025+0000 7fc302ffd640 1 -- 172.21.15.96:0/2395623168 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fc30800d330 con 0x7fc30c09b460 2024-03-21T06:39:04.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.199+0000 7fc3146dc640 1 -- 172.21.15.96:0/2395623168 --> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc30c130bc0 con 0x7fc2f0038260 2024-03-21T06:39:04.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.200+0000 7fc302ffd640 1 -- 172.21.15.96:0/2395623168 <== mgr.4099 v2:172.21.15.184:6800/2376803682 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (crc 0 0 0) 0x7fc2fc012490 con 0x7fc2f0038260 2024-03-21T06:39:04.199 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 3 to deep-scrub 2024-03-21T06:39:04.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.201+0000 7fc3146dc640 1 -- 172.21.15.96:0/2395623168 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc2f0038260 msgr2=0x7fc2f003a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:04.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.202+0000 7fc3146dc640 1 --2- 172.21.15.96:0/2395623168 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc2f0038260 0x7fc2f003a770 crc :-1 s=READY pgs=3873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:04.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.202+0000 7fc3146dc640 1 -- 172.21.15.96:0/2395623168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc30c09b460 msgr2=0x7fc30c1c5500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:04.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.202+0000 7fc3146dc640 1 --2- 172.21.15.96:0/2395623168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc30c09b460 0x7fc30c1c5500 crc :-1 s=READY pgs=13014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:04.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.202+0000 7fc3146dc640 1 -- 172.21.15.96:0/2395623168 shutdown_connections 2024-03-21T06:39:04.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.202+0000 7fc3146dc640 1 --2- 172.21.15.96:0/2395623168 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fc2f0038260 0x7fc2f003a770 unknown :-1 s=CLOSED pgs=3873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:04.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.202+0000 7fc3146dc640 1 --2- 172.21.15.96:0/2395623168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc30c09b460 0x7fc30c1c5500 unknown :-1 s=CLOSED pgs=13014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:04.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.202+0000 7fc3146dc640 1 -- 172.21.15.96:0/2395623168 >> 172.21.15.96:0/2395623168 conn(0x7fc30c09fdb0 msgr2=0x7fc30c126f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:04.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.202+0000 7fc3146dc640 1 -- 172.21.15.96:0/2395623168 shutdown_connections 2024-03-21T06:39:04.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.202+0000 7fc3146dc640 1 -- 172.21.15.96:0/2395623168 wait complete. 2024-03-21T06:39:04.212 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T06:39:04.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.314+0000 7fce0c2dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:04.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.318+0000 7fce0c2dc640 1 Processor -- start 2024-03-21T06:39:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.318+0000 7fce0c2dc640 1 -- start start 2024-03-21T06:39:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.318+0000 7fce0c2dc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fce04130b70 0x7fce04135990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.318+0000 7fce0c2dc640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce04099dc0 con 0x7fce0409b5d0 2024-03-21T06:39:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.319+0000 7fce0c2dc640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce0409bae0 con 0x7fce04130b70 2024-03-21T06:39:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.319+0000 7fce09877640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fce04130b70 0x7fce04135990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.319+0000 7fce0a078640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fce0409b5d0 0x7fce04130460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:53208/0 (socket says 172.21.15.96:53208) 2024-03-21T06:39:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.319+0000 7fce0a078640 1 -- 172.21.15.96:0/3074762954 learned_addr learned my addr 172.21.15.96:0/3074762954 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:39:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.319+0000 7fce09877640 1 -- 172.21.15.96:0/3074762954 >> v1:172.21.15.96:6789/0 conn(0x7fce0409b5d0 legacy=0x7fce04130460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.319+0000 7fce09877640 1 -- 172.21.15.96:0/3074762954 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0409c0f0 con 0x7fce04130b70 2024-03-21T06:39:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.319+0000 7fce09076640 1 -- 172.21.15.96:0/3074762954 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251856917 0 0) 0x7fce04099dc0 con 0x7fce0409b5d0 2024-03-21T06:39:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.319+0000 7fce09877640 1 --2- 172.21.15.96:0/3074762954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce04130b70 0x7fce04135990 crc :-1 s=READY pgs=13016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7943e99d4e2ee4a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:04.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.319+0000 7fce09076640 1 -- 172.21.15.96:0/3074762954 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcdf4009ce0 con 0x7fce04130b70 2024-03-21T06:39:04.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.319+0000 7fce09076640 1 -- 172.21.15.96:0/3074762954 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fcdf4009e40 con 0x7fce04130b70 2024-03-21T06:39:04.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.319+0000 7fce09076640 1 -- 172.21.15.96:0/3074762954 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fcdf400e360 con 0x7fce04130b70 2024-03-21T06:39:04.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.320+0000 7fce0c2dc640 1 -- 172.21.15.96:0/3074762954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce04130b70 msgr2=0x7fce04135990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:04.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.320+0000 7fce0c2dc640 1 --2- 172.21.15.96:0/3074762954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce04130b70 0x7fce04135990 crc :-1 s=READY pgs=13016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:04.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.320+0000 7fce0c2dc640 1 -- 172.21.15.96:0/3074762954 shutdown_connections 2024-03-21T06:39:04.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.320+0000 7fce0c2dc640 1 --2- 172.21.15.96:0/3074762954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce04130b70 0x7fce04135990 unknown :-1 s=CLOSED pgs=13016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:04.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.320+0000 7fce0c2dc640 1 -- 172.21.15.96:0/3074762954 >> 172.21.15.96:0/3074762954 conn(0x7fce0409fdb0 msgr2=0x7fce040a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:04.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.320+0000 7fce0c2dc640 1 -- 172.21.15.96:0/3074762954 shutdown_connections 2024-03-21T06:39:04.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.320+0000 7fce0c2dc640 1 -- 172.21.15.96:0/3074762954 wait complete. 2024-03-21T06:39:04.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.320+0000 7fce0c2dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:04.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.321+0000 7fce0c2dc640 1 Processor -- start 2024-03-21T06:39:04.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.321+0000 7fce0c2dc640 1 -- start start 2024-03-21T06:39:04.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.322+0000 7fce0c2dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce0409b5d0 0x7fce04071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:04.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.322+0000 7fce0c2dc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce040739d0 con 0x7fce0409b5d0 2024-03-21T06:39:04.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.322+0000 7fce0a078640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce0409b5d0 0x7fce04071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:04.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.322+0000 7fce0a078640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce0409b5d0 0x7fce04071cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56092/0 (socket says 172.21.15.96:56092) 2024-03-21T06:39:04.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.322+0000 7fce0a078640 1 -- 172.21.15.96:0/421856634 learned_addr learned my addr 172.21.15.96:0/421856634 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:39:04.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.322+0000 7fce0a078640 1 -- 172.21.15.96:0/421856634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce04128ad0 con 0x7fce0409b5d0 2024-03-21T06:39:04.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.322+0000 7fce0a078640 1 --2- 172.21.15.96:0/421856634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce0409b5d0 0x7fce04071cf0 crc :-1 s=READY pgs=13017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:04.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.322+0000 7fcdfaffd640 1 -- 172.21.15.96:0/421856634 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fce00004af0 con 0x7fce0409b5d0 2024-03-21T06:39:04.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.322+0000 7fcdfaffd640 1 -- 172.21.15.96:0/421856634 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fce00004c50 con 0x7fce0409b5d0 2024-03-21T06:39:04.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.322+0000 7fce0c2dc640 1 -- 172.21.15.96:0/421856634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce041273a0 con 0x7fce0409b5d0 2024-03-21T06:39:04.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.322+0000 7fcdfaffd640 1 -- 172.21.15.96:0/421856634 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fce00007120 con 0x7fce0409b5d0 2024-03-21T06:39:04.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.323+0000 7fce0c2dc640 1 -- 172.21.15.96:0/421856634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce04137650 con 0x7fce0409b5d0 2024-03-21T06:39:04.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.323+0000 7fce0c2dc640 1 -- 172.21.15.96:0/421856634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce04062f10 con 0x7fce0409b5d0 2024-03-21T06:39:04.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.324+0000 7fcdfaffd640 1 -- 172.21.15.96:0/421856634 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fce000072d0 con 0x7fce0409b5d0 2024-03-21T06:39:04.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.324+0000 7fcdfaffd640 1 --2- 172.21.15.96:0/421856634 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcde8038260 0x7fcde803a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:04.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.324+0000 7fcdfaffd640 1 -- 172.21.15.96:0/421856634 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(713..713 src has 402..713) v4 ==== 3403+0+0 (crc 0 0 0) 0x7fce00047bf0 con 0x7fce0409b5d0 2024-03-21T06:39:04.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.324+0000 7fce09877640 1 --2- 172.21.15.96:0/421856634 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcde8038260 0x7fcde803a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:04.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.325+0000 7fce09877640 1 --2- 172.21.15.96:0/421856634 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcde8038260 0x7fcde803a770 crc :-1 s=READY pgs=3874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:04.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.329+0000 7fcdfaffd640 1 -- 172.21.15.96:0/421856634 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fce0000a9e0 con 0x7fce0409b5d0 2024-03-21T06:39:04.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.498+0000 7fce0c2dc640 1 -- 172.21.15.96:0/421856634 --> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce0412fc20 con 0x7fcde8038260 2024-03-21T06:39:04.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.502+0000 7fcdfaffd640 1 -- 172.21.15.96:0/421856634 <== mgr.4099 v2:172.21.15.184:6800/2376803682 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+107144 (crc 0 0 0) 0x7fce0412fc20 con 0x7fcde8038260 2024-03-21T06:39:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:39:04.502 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-03-21T06:39:04.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.503+0000 7fce0c2dc640 1 -- 172.21.15.96:0/421856634 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcde8038260 msgr2=0x7fcde803a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:04.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.503+0000 7fce0c2dc640 1 --2- 172.21.15.96:0/421856634 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcde8038260 0x7fcde803a770 crc :-1 s=READY pgs=3874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:04.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.503+0000 7fce0c2dc640 1 -- 172.21.15.96:0/421856634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce0409b5d0 msgr2=0x7fce04071cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:04.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.503+0000 7fce0c2dc640 1 --2- 172.21.15.96:0/421856634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce0409b5d0 0x7fce04071cf0 crc :-1 s=READY pgs=13017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:04.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.503+0000 7fce0c2dc640 1 -- 172.21.15.96:0/421856634 shutdown_connections 2024-03-21T06:39:04.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.503+0000 7fce0c2dc640 1 --2- 172.21.15.96:0/421856634 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7fcde8038260 0x7fcde803a770 unknown :-1 s=CLOSED pgs=3874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:04.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.503+0000 7fce0c2dc640 1 --2- 172.21.15.96:0/421856634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce0409b5d0 0x7fce04071cf0 unknown :-1 s=CLOSED pgs=13017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:04.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.503+0000 7fce0c2dc640 1 -- 172.21.15.96:0/421856634 >> 172.21.15.96:0/421856634 conn(0x7fce0409fdb0 msgr2=0x7fce040a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:04.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.503+0000 7fce0c2dc640 1 -- 172.21.15.96:0/421856634 shutdown_connections 2024-03-21T06:39:04.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:04.503+0000 7fce0c2dc640 1 -- 172.21.15.96:0/421856634 wait complete. 2024-03-21T06:39:04.513 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":1268,"stamp":"2024-03-21T06:39:02.860589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":288232,"kb_used_data":15240,"kb_used_omap":0,"kb_used_meta":272960,"kb_avail":419142168,"statfs":{"total":429496729600,"available":429201580032,"internally_reserved":0,"allocated":15605760,"data_stored":6956825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279511040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-295300,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-25,"num_write_kb":-300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002085"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.760844+0000","last_change":"2024-03-21T06:25:58.485657+0000","last_active":"2024-03-21T06:38:29.760844+0000","last_peered":"2024-03-21T06:38:29.760844+0000","last_clean":"2024-03-21T06:38:29.760844+0000","last_became_active":"2024-03-21T06:25:58.480248+0000","last_became_peered":"2024-03-21T06:25:58.480248+0000","last_unstale":"2024-03-21T06:38:29.760844+0000","last_undegraded":"2024-03-21T06:38:29.760844+0000","last_fullsized":"2024-03-21T06:38:29.760844+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.500365+0000","last_change":"2024-03-21T06:25:58.492610+0000","last_active":"2024-03-21T06:38:58.500365+0000","last_peered":"2024-03-21T06:38:58.500365+0000","last_clean":"2024-03-21T06:38:58.500365+0000","last_became_active":"2024-03-21T06:25:58.484415+0000","last_became_peered":"2024-03-21T06:25:58.484415+0000","last_unstale":"2024-03-21T06:38:58.500365+0000","last_undegraded":"2024-03-21T06:38:58.500365+0000","last_fullsized":"2024-03-21T06:38:58.500365+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.521005+0000","last_change":"2024-03-21T06:25:58.492051+0000","last_active":"2024-03-21T06:38:27.521005+0000","last_peered":"2024-03-21T06:38:27.521005+0000","last_clean":"2024-03-21T06:38:27.521005+0000","last_became_active":"2024-03-21T06:25:58.487475+0000","last_became_peered":"2024-03-21T06:25:58.487475+0000","last_unstale":"2024-03-21T06:38:27.521005+0000","last_undegraded":"2024-03-21T06:38:27.521005+0000","last_fullsized":"2024-03-21T06:38:27.521005+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.713728+0000","last_change":"2024-03-21T06:25:58.528878+0000","last_active":"2024-03-21T06:38:29.713728+0000","last_peered":"2024-03-21T06:38:29.713728+0000","last_clean":"2024-03-21T06:38:29.713728+0000","last_became_active":"2024-03-21T06:25:58.519468+0000","last_became_peered":"2024-03-21T06:25:58.519468+0000","last_unstale":"2024-03-21T06:38:29.713728+0000","last_undegraded":"2024-03-21T06:38:29.713728+0000","last_fullsized":"2024-03-21T06:38:29.713728+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.747889+0000","last_change":"2024-03-21T06:25:58.530972+0000","last_active":"2024-03-21T06:38:29.747889+0000","last_peered":"2024-03-21T06:38:29.747889+0000","last_clean":"2024-03-21T06:38:29.747889+0000","last_became_active":"2024-03-21T06:25:58.527143+0000","last_became_peered":"2024-03-21T06:25:58.527143+0000","last_unstale":"2024-03-21T06:38:29.747889+0000","last_undegraded":"2024-03-21T06:38:29.747889+0000","last_fullsized":"2024-03-21T06:38:29.747889+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.483335+0000","last_change":"2024-03-21T06:25:58.506594+0000","last_active":"2024-03-21T06:38:58.483335+0000","last_peered":"2024-03-21T06:38:58.483335+0000","last_clean":"2024-03-21T06:38:58.483335+0000","last_became_active":"2024-03-21T06:25:58.494870+0000","last_became_peered":"2024-03-21T06:25:58.494870+0000","last_unstale":"2024-03-21T06:38:58.483335+0000","last_undegraded":"2024-03-21T06:38:58.483335+0000","last_fullsized":"2024-03-21T06:38:58.483335+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.530393+0000","last_change":"2024-03-21T06:25:58.500230+0000","last_active":"2024-03-21T06:38:58.530393+0000","last_peered":"2024-03-21T06:38:58.530393+0000","last_clean":"2024-03-21T06:38:58.530393+0000","last_became_active":"2024-03-21T06:25:58.488027+0000","last_became_peered":"2024-03-21T06:25:58.488027+0000","last_unstale":"2024-03-21T06:38:58.530393+0000","last_undegraded":"2024-03-21T06:38:58.530393+0000","last_fullsized":"2024-03-21T06:38:58.530393+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.488223+0000","last_change":"2024-03-21T06:25:58.505580+0000","last_active":"2024-03-21T06:38:58.488223+0000","last_peered":"2024-03-21T06:38:58.488223+0000","last_clean":"2024-03-21T06:38:58.488223+0000","last_became_active":"2024-03-21T06:25:58.498450+0000","last_became_peered":"2024-03-21T06:25:58.498450+0000","last_unstale":"2024-03-21T06:38:58.488223+0000","last_undegraded":"2024-03-21T06:38:58.488223+0000","last_fullsized":"2024-03-21T06:38:58.488223+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.720009+0000","last_change":"2024-03-21T06:25:58.515242+0000","last_active":"2024-03-21T06:38:29.720009+0000","last_peered":"2024-03-21T06:38:29.720009+0000","last_clean":"2024-03-21T06:38:29.720009+0000","last_became_active":"2024-03-21T06:25:58.503376+0000","last_became_peered":"2024-03-21T06:25:58.503376+0000","last_unstale":"2024-03-21T06:38:29.720009+0000","last_undegraded":"2024-03-21T06:38:29.720009+0000","last_fullsized":"2024-03-21T06:38:29.720009+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.517006+0000","last_change":"2024-03-21T06:25:58.500753+0000","last_active":"2024-03-21T06:38:27.517006+0000","last_peered":"2024-03-21T06:38:27.517006+0000","last_clean":"2024-03-21T06:38:27.517006+0000","last_became_active":"2024-03-21T06:25:58.497498+0000","last_became_peered":"2024-03-21T06:25:58.497498+0000","last_unstale":"2024-03-21T06:38:27.517006+0000","last_undegraded":"2024-03-21T06:38:27.517006+0000","last_fullsized":"2024-03-21T06:38:27.517006+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.695561+0000","last_change":"2024-03-21T06:25:58.509601+0000","last_active":"2024-03-21T06:38:29.695561+0000","last_peered":"2024-03-21T06:38:29.695561+0000","last_clean":"2024-03-21T06:38:29.695561+0000","last_became_active":"2024-03-21T06:25:58.498895+0000","last_became_peered":"2024-03-21T06:25:58.498895+0000","last_unstale":"2024-03-21T06:38:29.695561+0000","last_undegraded":"2024-03-21T06:38:29.695561+0000","last_fullsized":"2024-03-21T06:38:29.695561+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.505272+0000","last_change":"2024-03-21T06:25:58.496259+0000","last_active":"2024-03-21T06:38:58.505272+0000","last_peered":"2024-03-21T06:38:58.505272+0000","last_clean":"2024-03-21T06:38:58.505272+0000","last_became_active":"2024-03-21T06:25:58.486238+0000","last_became_peered":"2024-03-21T06:25:58.486238+0000","last_unstale":"2024-03-21T06:38:58.505272+0000","last_undegraded":"2024-03-21T06:38:58.505272+0000","last_fullsized":"2024-03-21T06:38:58.505272+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.711452+0000","last_change":"2024-03-21T06:25:58.524275+0000","last_active":"2024-03-21T06:38:29.711452+0000","last_peered":"2024-03-21T06:38:29.711452+0000","last_clean":"2024-03-21T06:38:29.711452+0000","last_became_active":"2024-03-21T06:25:58.513806+0000","last_became_peered":"2024-03-21T06:25:58.513806+0000","last_unstale":"2024-03-21T06:38:29.711452+0000","last_undegraded":"2024-03-21T06:38:29.711452+0000","last_fullsized":"2024-03-21T06:38:29.711452+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.707100+0000","last_change":"2024-03-21T06:25:58.540399+0000","last_active":"2024-03-21T06:38:29.707100+0000","last_peered":"2024-03-21T06:38:29.707100+0000","last_clean":"2024-03-21T06:38:29.707100+0000","last_became_active":"2024-03-21T06:25:58.537009+0000","last_became_peered":"2024-03-21T06:25:58.537009+0000","last_unstale":"2024-03-21T06:38:29.707100+0000","last_undegraded":"2024-03-21T06:38:29.707100+0000","last_fullsized":"2024-03-21T06:38:29.707100+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.482596+0000","last_change":"2024-03-21T06:25:58.514471+0000","last_active":"2024-03-21T06:38:58.482596+0000","last_peered":"2024-03-21T06:38:58.482596+0000","last_clean":"2024-03-21T06:38:58.482596+0000","last_became_active":"2024-03-21T06:25:58.510651+0000","last_became_peered":"2024-03-21T06:25:58.510651+0000","last_unstale":"2024-03-21T06:38:58.482596+0000","last_undegraded":"2024-03-21T06:38:58.482596+0000","last_fullsized":"2024-03-21T06:38:58.482596+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.529785+0000","last_change":"2024-03-21T06:25:58.551761+0000","last_active":"2024-03-21T06:38:58.529785+0000","last_peered":"2024-03-21T06:38:58.529785+0000","last_clean":"2024-03-21T06:38:58.529785+0000","last_became_active":"2024-03-21T06:25:58.548956+0000","last_became_peered":"2024-03-21T06:25:58.548956+0000","last_unstale":"2024-03-21T06:38:58.529785+0000","last_undegraded":"2024-03-21T06:38:58.529785+0000","last_fullsized":"2024-03-21T06:38:58.529785+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.499863+0000","last_change":"2024-03-21T06:25:58.490344+0000","last_active":"2024-03-21T06:38:58.499863+0000","last_peered":"2024-03-21T06:38:58.499863+0000","last_clean":"2024-03-21T06:38:58.499863+0000","last_became_active":"2024-03-21T06:25:58.482358+0000","last_became_peered":"2024-03-21T06:25:58.482358+0000","last_unstale":"2024-03-21T06:38:58.499863+0000","last_undegraded":"2024-03-21T06:38:58.499863+0000","last_fullsized":"2024-03-21T06:38:58.499863+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.718502+0000","last_change":"2024-03-21T06:25:58.527681+0000","last_active":"2024-03-21T06:38:29.718502+0000","last_peered":"2024-03-21T06:38:29.718502+0000","last_clean":"2024-03-21T06:38:29.718502+0000","last_became_active":"2024-03-21T06:25:58.521822+0000","last_became_peered":"2024-03-21T06:25:58.521822+0000","last_unstale":"2024-03-21T06:38:29.718502+0000","last_undegraded":"2024-03-21T06:38:29.718502+0000","last_fullsized":"2024-03-21T06:38:29.718502+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.526661+0000","last_change":"2024-03-21T06:25:58.490537+0000","last_active":"2024-03-21T06:38:27.526661+0000","last_peered":"2024-03-21T06:38:27.526661+0000","last_clean":"2024-03-21T06:38:27.526661+0000","last_became_active":"2024-03-21T06:25:58.484498+0000","last_became_peered":"2024-03-21T06:25:58.484498+0000","last_unstale":"2024-03-21T06:38:27.526661+0000","last_undegraded":"2024-03-21T06:38:27.526661+0000","last_fullsized":"2024-03-21T06:38:27.526661+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.734123+0000","last_change":"2024-03-21T06:25:58.482707+0000","last_active":"2024-03-21T06:38:29.734123+0000","last_peered":"2024-03-21T06:38:29.734123+0000","last_clean":"2024-03-21T06:38:29.734123+0000","last_became_active":"2024-03-21T06:25:58.474376+0000","last_became_peered":"2024-03-21T06:25:58.474376+0000","last_unstale":"2024-03-21T06:38:29.734123+0000","last_undegraded":"2024-03-21T06:38:29.734123+0000","last_fullsized":"2024-03-21T06:38:29.734123+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.754301+0000","last_change":"2024-03-21T06:25:58.483801+0000","last_active":"2024-03-21T06:38:29.754301+0000","last_peered":"2024-03-21T06:38:29.754301+0000","last_clean":"2024-03-21T06:38:29.754301+0000","last_became_active":"2024-03-21T06:25:58.477107+0000","last_became_peered":"2024-03-21T06:25:58.477107+0000","last_unstale":"2024-03-21T06:38:29.754301+0000","last_undegraded":"2024-03-21T06:38:29.754301+0000","last_fullsized":"2024-03-21T06:38:29.754301+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1271,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.736832+0000","last_change":"2024-03-21T06:25:58.494316+0000","last_active":"2024-03-21T06:38:29.736832+0000","last_peered":"2024-03-21T06:38:29.736832+0000","last_clean":"2024-03-21T06:38:29.736832+0000","last_became_active":"2024-03-21T06:25:58.490894+0000","last_became_peered":"2024-03-21T06:25:58.490894+0000","last_unstale":"2024-03-21T06:38:29.736832+0000","last_undegraded":"2024-03-21T06:38:29.736832+0000","last_fullsized":"2024-03-21T06:38:29.736832+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.523400+0000","last_change":"2024-03-21T06:25:58.508102+0000","last_active":"2024-03-21T06:38:27.523400+0000","last_peered":"2024-03-21T06:38:27.523400+0000","last_clean":"2024-03-21T06:38:27.523400+0000","last_became_active":"2024-03-21T06:25:58.505636+0000","last_became_peered":"2024-03-21T06:25:58.505636+0000","last_unstale":"2024-03-21T06:38:27.523400+0000","last_undegraded":"2024-03-21T06:38:27.523400+0000","last_fullsized":"2024-03-21T06:38:27.523400+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.714562+0000","last_change":"2024-03-21T06:25:58.512176+0000","last_active":"2024-03-21T06:38:29.714562+0000","last_peered":"2024-03-21T06:38:29.714562+0000","last_clean":"2024-03-21T06:38:29.714562+0000","last_became_active":"2024-03-21T06:25:58.501608+0000","last_became_peered":"2024-03-21T06:25:58.501608+0000","last_unstale":"2024-03-21T06:38:29.714562+0000","last_undegraded":"2024-03-21T06:38:29.714562+0000","last_fullsized":"2024-03-21T06:38:29.714562+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.715197+0000","last_change":"2024-03-21T06:25:58.517908+0000","last_active":"2024-03-21T06:38:29.715197+0000","last_peered":"2024-03-21T06:38:29.715197+0000","last_clean":"2024-03-21T06:38:29.715197+0000","last_became_active":"2024-03-21T06:25:58.508192+0000","last_became_peered":"2024-03-21T06:25:58.508192+0000","last_unstale":"2024-03-21T06:38:29.715197+0000","last_undegraded":"2024-03-21T06:38:29.715197+0000","last_fullsized":"2024-03-21T06:38:29.715197+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.517104+0000","last_change":"2024-03-21T06:25:58.501302+0000","last_active":"2024-03-21T06:38:58.517104+0000","last_peered":"2024-03-21T06:38:58.517104+0000","last_clean":"2024-03-21T06:38:58.517104+0000","last_became_active":"2024-03-21T06:25:58.491525+0000","last_became_peered":"2024-03-21T06:25:58.491525+0000","last_unstale":"2024-03-21T06:38:58.517104+0000","last_undegraded":"2024-03-21T06:38:58.517104+0000","last_fullsized":"2024-03-21T06:38:58.517104+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1261,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.700998+0000","last_change":"2024-03-21T06:25:58.516179+0000","last_active":"2024-03-21T06:38:29.700998+0000","last_peered":"2024-03-21T06:38:29.700998+0000","last_clean":"2024-03-21T06:38:29.700998+0000","last_became_active":"2024-03-21T06:25:58.506657+0000","last_became_peered":"2024-03-21T06:25:58.506657+0000","last_unstale":"2024-03-21T06:38:29.700998+0000","last_undegraded":"2024-03-21T06:38:29.700998+0000","last_fullsized":"2024-03-21T06:38:29.700998+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.490947+0000","last_change":"2024-03-21T06:25:58.521034+0000","last_active":"2024-03-21T06:38:58.490947+0000","last_peered":"2024-03-21T06:38:58.490947+0000","last_clean":"2024-03-21T06:38:58.490947+0000","last_became_active":"2024-03-21T06:25:58.517030+0000","last_became_peered":"2024-03-21T06:25:58.517030+0000","last_unstale":"2024-03-21T06:38:58.490947+0000","last_undegraded":"2024-03-21T06:38:58.490947+0000","last_fullsized":"2024-03-21T06:38:58.490947+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.493691+0000","last_change":"2024-03-21T06:25:58.531640+0000","last_active":"2024-03-21T06:38:58.493691+0000","last_peered":"2024-03-21T06:38:58.493691+0000","last_clean":"2024-03-21T06:38:58.493691+0000","last_became_active":"2024-03-21T06:25:58.528601+0000","last_became_peered":"2024-03-21T06:25:58.528601+0000","last_unstale":"2024-03-21T06:38:58.493691+0000","last_undegraded":"2024-03-21T06:38:58.493691+0000","last_fullsized":"2024-03-21T06:38:58.493691+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1274,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.532701+0000","last_change":"2024-03-21T06:25:58.509341+0000","last_active":"2024-03-21T06:38:58.532701+0000","last_peered":"2024-03-21T06:38:58.532701+0000","last_clean":"2024-03-21T06:38:58.532701+0000","last_became_active":"2024-03-21T06:25:58.503544+0000","last_became_peered":"2024-03-21T06:25:58.503544+0000","last_unstale":"2024-03-21T06:38:58.532701+0000","last_undegraded":"2024-03-21T06:38:58.532701+0000","last_fullsized":"2024-03-21T06:38:58.532701+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"236'7","reported_seq":1284,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.757963+0000","last_change":"2024-03-21T06:25:58.509187+0000","last_active":"2024-03-21T06:38:29.757963+0000","last_peered":"2024-03-21T06:38:29.757963+0000","last_clean":"2024-03-21T06:38:29.757963+0000","last_became_active":"2024-03-21T06:25:58.505585+0000","last_became_peered":"2024-03-21T06:25:58.505585+0000","last_unstale":"2024-03-21T06:38:29.757963+0000","last_undegraded":"2024-03-21T06:38:29.757963+0000","last_fullsized":"2024-03-21T06:38:29.757963+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1281,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.513968+0000","last_change":"2024-03-21T06:25:58.486325+0000","last_active":"2024-03-21T06:38:27.513968+0000","last_peered":"2024-03-21T06:38:27.513968+0000","last_clean":"2024-03-21T06:38:27.513968+0000","last_became_active":"2024-03-21T06:25:58.481996+0000","last_became_peered":"2024-03-21T06:25:58.481996+0000","last_unstale":"2024-03-21T06:38:27.513968+0000","last_undegraded":"2024-03-21T06:38:27.513968+0000","last_fullsized":"2024-03-21T06:38:27.513968+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:25:57.372019+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1870,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.736177+0000","last_change":"2024-03-21T06:20:23.724343+0000","last_active":"2024-03-21T06:38:29.736177+0000","last_peered":"2024-03-21T06:38:29.736177+0000","last_clean":"2024-03-21T06:38:29.736177+0000","last_became_active":"2024-03-21T06:20:23.721083+0000","last_became_peered":"2024-03-21T06:20:23.721083+0000","last_unstale":"2024-03-21T06:38:29.736177+0000","last_undegraded":"2024-03-21T06:38:29.736177+0000","last_fullsized":"2024-03-21T06:38:29.736177+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1880,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.608992+0000","last_change":"2024-03-21T06:20:23.720094+0000","last_active":"2024-03-21T06:38:27.608992+0000","last_peered":"2024-03-21T06:38:27.608992+0000","last_clean":"2024-03-21T06:38:27.608992+0000","last_became_active":"2024-03-21T06:20:23.717296+0000","last_became_peered":"2024-03-21T06:20:23.717296+0000","last_unstale":"2024-03-21T06:38:27.608992+0000","last_undegraded":"2024-03-21T06:38:27.608992+0000","last_fullsized":"2024-03-21T06:38:27.608992+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1864,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.785694+0000","last_change":"2024-03-21T06:20:23.724647+0000","last_active":"2024-03-21T06:38:29.785694+0000","last_peered":"2024-03-21T06:38:29.785694+0000","last_clean":"2024-03-21T06:38:29.785694+0000","last_became_active":"2024-03-21T06:20:23.721198+0000","last_became_peered":"2024-03-21T06:20:23.721198+0000","last_unstale":"2024-03-21T06:38:29.785694+0000","last_undegraded":"2024-03-21T06:38:29.785694+0000","last_fullsized":"2024-03-21T06:38:29.785694+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1880,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:27.522321+0000","last_change":"2024-03-21T06:20:23.716953+0000","last_active":"2024-03-21T06:38:27.522321+0000","last_peered":"2024-03-21T06:38:27.522321+0000","last_clean":"2024-03-21T06:38:27.522321+0000","last_became_active":"2024-03-21T06:20:23.714274+0000","last_became_peered":"2024-03-21T06:20:23.714274+0000","last_unstale":"2024-03-21T06:38:27.522321+0000","last_undegraded":"2024-03-21T06:38:27.522321+0000","last_fullsized":"2024-03-21T06:38:27.522321+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1867,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.546801+0000","last_change":"2024-03-21T06:20:23.717551+0000","last_active":"2024-03-21T06:38:58.546801+0000","last_peered":"2024-03-21T06:38:58.546801+0000","last_clean":"2024-03-21T06:38:58.546801+0000","last_became_active":"2024-03-21T06:20:23.714766+0000","last_became_peered":"2024-03-21T06:20:23.714766+0000","last_unstale":"2024-03-21T06:38:58.546801+0000","last_undegraded":"2024-03-21T06:38:58.546801+0000","last_fullsized":"2024-03-21T06:38:58.546801+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1870,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.759598+0000","last_change":"2024-03-21T06:20:23.716172+0000","last_active":"2024-03-21T06:38:29.759598+0000","last_peered":"2024-03-21T06:38:29.759598+0000","last_clean":"2024-03-21T06:38:29.759598+0000","last_became_active":"2024-03-21T06:20:23.712781+0000","last_became_peered":"2024-03-21T06:20:23.712781+0000","last_unstale":"2024-03-21T06:38:29.759598+0000","last_undegraded":"2024-03-21T06:38:29.759598+0000","last_fullsized":"2024-03-21T06:38:29.759598+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1867,"reported_epoch":712,"state":"active+clean","last_fresh":"2024-03-21T06:38:58.557390+0000","last_change":"2024-03-21T06:20:23.724311+0000","last_active":"2024-03-21T06:38:58.557390+0000","last_peered":"2024-03-21T06:38:58.557390+0000","last_clean":"2024-03-21T06:38:58.557390+0000","last_became_active":"2024-03-21T06:20:23.720894+0000","last_became_peered":"2024-03-21T06:20:23.720894+0000","last_unstale":"2024-03-21T06:38:58.557390+0000","last_undegraded":"2024-03-21T06:38:58.557390+0000","last_fullsized":"2024-03-21T06:38:58.557390+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1870,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.809884+0000","last_change":"2024-03-21T06:20:23.720175+0000","last_active":"2024-03-21T06:38:29.809884+0000","last_peered":"2024-03-21T06:38:29.809884+0000","last_clean":"2024-03-21T06:38:29.809884+0000","last_became_active":"2024-03-21T06:20:23.716768+0000","last_became_peered":"2024-03-21T06:20:23.716768+0000","last_unstale":"2024-03-21T06:38:29.809884+0000","last_undegraded":"2024-03-21T06:38:29.809884+0000","last_fullsized":"2024-03-21T06:38:29.809884+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:22.674032+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":1997,"reported_epoch":711,"state":"active+clean","last_fresh":"2024-03-21T06:38:29.703106+0000","last_change":"2024-03-21T06:20:21.697506+0000","last_active":"2024-03-21T06:38:29.703106+0000","last_peered":"2024-03-21T06:38:29.703106+0000","last_clean":"2024-03-21T06:38:29.703106+0000","last_became_active":"2024-03-21T06:20:21.690580+0000","last_became_peered":"2024-03-21T06:20:21.690580+0000","last_unstale":"2024-03-21T06:38:29.703106+0000","last_undegraded":"2024-03-21T06:38:29.703106+0000","last_fullsized":"2024-03-21T06:38:29.703106+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:20:20.658895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:20:20.658895+0000","last_clean_scrub_stamp":"2024-03-21T06:20:20.658895+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":71252,"kb_used_data":4044,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":104786348,"statfs":{"total":107374182400,"available":107301220352,"internally_reserved":0,"allocated":4141056,"data_stored":2006883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":70756,"kb_used_data":3548,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":104786844,"statfs":{"total":107374182400,"available":107301728256,"internally_reserved":0,"allocated":3633152,"data_stored":1449822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":70740,"kb_used_data":3532,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":104786860,"statfs":{"total":107374182400,"available":107301744640,"internally_reserved":0,"allocated":3616768,"data_stored":1448988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":75484,"kb_used_data":4116,"kb_used_omap":0,"kb_used_meta":71360,"kb_avail":104782116,"statfs":{"total":107374182400,"available":107296886784,"internally_reserved":0,"allocated":4214784,"data_stored":2051132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":73072640},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-21T06:39:04.521 INFO:tasks.ceph:pgid 75.1f last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.521 INFO:tasks.ceph:pgid 75.1e last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.521 INFO:tasks.ceph:pgid 75.1d last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.522 INFO:tasks.ceph:pgid 75.1c last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.522 INFO:tasks.ceph:pgid 75.1b last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.522 INFO:tasks.ceph:pgid 75.1a last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.522 INFO:tasks.ceph:pgid 75.19 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.522 INFO:tasks.ceph:pgid 75.18 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.523 INFO:tasks.ceph:pgid 75.17 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.523 INFO:tasks.ceph:pgid 75.16 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.523 INFO:tasks.ceph:pgid 75.15 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.523 INFO:tasks.ceph:pgid 75.14 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.524 INFO:tasks.ceph:pgid 75.13 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.524 INFO:tasks.ceph:pgid 75.12 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.524 INFO:tasks.ceph:pgid 75.11 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.524 INFO:tasks.ceph:pgid 75.10 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.524 INFO:tasks.ceph:pgid 75.f last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.524 INFO:tasks.ceph:pgid 75.e last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.524 INFO:tasks.ceph:pgid 75.d last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.525 INFO:tasks.ceph:pgid 75.c last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.525 INFO:tasks.ceph:pgid 75.b last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.525 INFO:tasks.ceph:pgid 75.a last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.525 INFO:tasks.ceph:pgid 75.9 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.525 INFO:tasks.ceph:pgid 75.8 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.525 INFO:tasks.ceph:pgid 75.7 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.525 INFO:tasks.ceph:pgid 75.6 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.525 INFO:tasks.ceph:pgid 75.5 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.525 INFO:tasks.ceph:pgid 75.4 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.526 INFO:tasks.ceph:pgid 75.3 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.526 INFO:tasks.ceph:pgid 75.2 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.526 INFO:tasks.ceph:pgid 75.1 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.526 INFO:tasks.ceph:pgid 75.0 last_scrub_stamp 2024-03-21T06:25:57.372019+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=25, tm_sec=57, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.526 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-03-21T06:20:22.674032+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=20, tm_sec=22, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.526 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-03-21T06:20:22.674032+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=20, tm_sec=22, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.526 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-03-21T06:20:22.674032+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=20, tm_sec=22, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.527 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-03-21T06:20:22.674032+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=20, tm_sec=22, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.527 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-03-21T06:20:22.674032+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=20, tm_sec=22, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.527 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-03-21T06:20:22.674032+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=20, tm_sec=22, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.527 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-03-21T06:20:22.674032+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=20, tm_sec=22, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.527 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-03-21T06:20:22.674032+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=20, tm_sec=22, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.527 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-21T06:20:20.658895+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=20, tm_sec=20, tm_wday=3, tm_yday=81, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=21, tm_hour=6, tm_min=39, tm_sec=1, tm_wday=3, tm_yday=81, tm_isdst=0) 2024-03-21T06:39:04.527 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-21T06:39:05.489 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-03-21T06:39:05.489+0000 7f134dff6640 -1 log_channel(cluster) log [ERR] : Health check failed: 23 scrub errors (OSD_SCRUB_ERRORS) 2024-03-21T06:39:05.489 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-03-21T06:39:05.489+0000 7f134dff6640 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 23 pgs inconsistent (PG_DAMAGED) 2024-03-21T06:39:12.468 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-03-21T06:39:12.469+0000 7f134dff6640 -1 log_channel(cluster) log [ERR] : Health check update: 41 scrub errors (OSD_SCRUB_ERRORS) 2024-03-21T06:39:12.469 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-03-21T06:39:12.469+0000 7f134dff6640 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 41 pgs inconsistent (PG_DAMAGED) 2024-03-21T06:39:16.678 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:39:16.679+0000 7fd0a88a9640 -1 calc_pg_upmaps abort due to max <= 0 2024-03-21T06:39:16.679 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:39:16.679+0000 7fd0a88a9640 -1 calc_pg_upmaps abort due to max <= 0 2024-03-21T06:39:16.679 INFO:tasks.ceph.mgr.x.smithi184.stderr:2024-03-21T06:39:16.679+0000 7fd0a88a9640 -1 calc_pg_upmaps abort due to max <= 0 2024-03-21T06:39:24.529 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T06:39:24.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.645+0000 7f306f11c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:24.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.649+0000 7f306f11c640 1 Processor -- start 2024-03-21T06:39:24.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.650+0000 7f306f11c640 1 -- start start 2024-03-21T06:39:24.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.650+0000 7f306f11c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f306809b360 0x7f306809cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:24.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.650+0000 7f306f11c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306806e410 con 0x7f30681315d0 2024-03-21T06:39:24.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.650+0000 7f306f11c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306809b770 con 0x7f306809b360 2024-03-21T06:39:24.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.650+0000 7f306ceb8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f306809b360 0x7f306809cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:24.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.650+0000 7f3067fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f30681315d0 0x7f3068134130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33868/0 (socket says 172.21.15.96:33868) 2024-03-21T06:39:24.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.650+0000 7f3067fff640 1 -- 172.21.15.96:0/3155023181 learned_addr learned my addr 172.21.15.96:0/3155023181 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:39:24.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.651+0000 7f306ceb8640 1 -- 172.21.15.96:0/3155023181 >> v1:172.21.15.96:6789/0 conn(0x7f30681315d0 legacy=0x7f3068134130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:24.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.651+0000 7f306ceb8640 1 -- 172.21.15.96:0/3155023181 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306806dd00 con 0x7f306809b360 2024-03-21T06:39:24.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.651+0000 7f30677fe640 1 -- 172.21.15.96:0/3155023181 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3719014641 0 0) 0x7f306806e410 con 0x7f30681315d0 2024-03-21T06:39:24.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.651+0000 7f306ceb8640 1 --2- 172.21.15.96:0/3155023181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f306809b360 0x7f306809cda0 crc :-1 s=READY pgs=13019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c173478be8fd6ea server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:24.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.651+0000 7f30677fe640 1 -- 172.21.15.96:0/3155023181 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f306000e040 con 0x7f306809b360 2024-03-21T06:39:24.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.651+0000 7f30677fe640 1 -- 172.21.15.96:0/3155023181 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3060007670 con 0x7f306809b360 2024-03-21T06:39:24.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.651+0000 7f30677fe640 1 -- 172.21.15.96:0/3155023181 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3060007910 con 0x7f306809b360 2024-03-21T06:39:24.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.652+0000 7f306f11c640 1 -- 172.21.15.96:0/3155023181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f306809b360 msgr2=0x7f306809cda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:24.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.652+0000 7f306f11c640 1 --2- 172.21.15.96:0/3155023181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f306809b360 0x7f306809cda0 crc :-1 s=READY pgs=13019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:24.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.652+0000 7f306f11c640 1 -- 172.21.15.96:0/3155023181 shutdown_connections 2024-03-21T06:39:24.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.652+0000 7f306f11c640 1 --2- 172.21.15.96:0/3155023181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f306809b360 0x7f306809cda0 unknown :-1 s=CLOSED pgs=13019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:24.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.652+0000 7f306f11c640 1 -- 172.21.15.96:0/3155023181 >> 172.21.15.96:0/3155023181 conn(0x7f30680a42c0 msgr2=0x7f30680a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:24.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.652+0000 7f306f11c640 1 -- 172.21.15.96:0/3155023181 shutdown_connections 2024-03-21T06:39:24.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.652+0000 7f306f11c640 1 -- 172.21.15.96:0/3155023181 wait complete. 2024-03-21T06:39:24.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.652+0000 7f306f11c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:24.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.653+0000 7f306f11c640 1 Processor -- start 2024-03-21T06:39:24.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.653+0000 7f306f11c640 1 -- start start 2024-03-21T06:39:24.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.654+0000 7f306f11c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f306809b360 0x7f306811f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:24.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.654+0000 7f306f11c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3068130780 con 0x7f306809b360 2024-03-21T06:39:24.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.654+0000 7f306ceb8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f306809b360 0x7f306811f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:24.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.654+0000 7f306ceb8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f306809b360 0x7f306811f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43800/0 (socket says 172.21.15.96:43800) 2024-03-21T06:39:24.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.654+0000 7f306ceb8640 1 -- 172.21.15.96:0/2410804833 learned_addr learned my addr 172.21.15.96:0/2410804833 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:39:24.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.654+0000 7f306ceb8640 1 -- 172.21.15.96:0/2410804833 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3068130a00 con 0x7f306809b360 2024-03-21T06:39:24.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.654+0000 7f306ceb8640 1 --2- 172.21.15.96:0/2410804833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f306809b360 0x7f306811f740 crc :-1 s=READY pgs=13020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:24.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.654+0000 7f30657fa640 1 -- 172.21.15.96:0/2410804833 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f306000e040 con 0x7f306809b360 2024-03-21T06:39:24.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.654+0000 7f30657fa640 1 -- 172.21.15.96:0/2410804833 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7f3060012b00 con 0x7f306809b360 2024-03-21T06:39:24.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.654+0000 7f306f11c640 1 -- 172.21.15.96:0/2410804833 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3068072fc0 con 0x7f306809b360 2024-03-21T06:39:24.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.654+0000 7f30657fa640 1 -- 172.21.15.96:0/2410804833 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7f3060012da0 con 0x7f306809b360 2024-03-21T06:39:24.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.654+0000 7f306f11c640 1 -- 172.21.15.96:0/2410804833 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3068121c00 con 0x7f306809b360 2024-03-21T06:39:24.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.655+0000 7f30657fa640 1 -- 172.21.15.96:0/2410804833 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7f30600153f0 con 0x7f306809b360 2024-03-21T06:39:24.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.655+0000 7f306f11c640 1 -- 172.21.15.96:0/2410804833 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306809ba50 con 0x7f306809b360 2024-03-21T06:39:24.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.655+0000 7f30657fa640 1 --2- 172.21.15.96:0/2410804833 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3048038260 0x7f304803a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:24.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.655+0000 7f30657fa640 1 -- 172.21.15.96:0/2410804833 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(713..713 src has 402..713) v4 ==== 3403+0+0 (crc 0 0 0) 0x7f3060049460 con 0x7f306809b360 2024-03-21T06:39:24.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.656+0000 7f3067fff640 1 --2- 172.21.15.96:0/2410804833 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3048038260 0x7f304803a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:24.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.656+0000 7f3067fff640 1 --2- 172.21.15.96:0/2410804833 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3048038260 0x7f304803a770 crc :-1 s=READY pgs=3875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:24.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.660+0000 7f30657fa640 1 -- 172.21.15.96:0/2410804833 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7f306000fc40 con 0x7f306809b360 2024-03-21T06:39:24.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.835+0000 7f306f11c640 1 -- 172.21.15.96:0/2410804833 --> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3068062f10 con 0x7f3048038260 2024-03-21T06:39:24.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.838+0000 7f30657fa640 1 -- 172.21.15.96:0/2410804833 <== mgr.4099 v2:172.21.15.184:6800/2376803682 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+107671 (crc 0 0 0) 0x7f3068062f10 con 0x7f3048038260 2024-03-21T06:39:24.838 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-21T06:39:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-03-21T06:39:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.840+0000 7f306f11c640 1 -- 172.21.15.96:0/2410804833 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3048038260 msgr2=0x7f304803a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.840+0000 7f306f11c640 1 --2- 172.21.15.96:0/2410804833 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3048038260 0x7f304803a770 crc :-1 s=READY pgs=3875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.840+0000 7f306f11c640 1 -- 172.21.15.96:0/2410804833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f306809b360 msgr2=0x7f306811f740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.840+0000 7f306f11c640 1 --2- 172.21.15.96:0/2410804833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f306809b360 0x7f306811f740 crc :-1 s=READY pgs=13020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.840+0000 7f306f11c640 1 -- 172.21.15.96:0/2410804833 shutdown_connections 2024-03-21T06:39:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.840+0000 7f306f11c640 1 --2- 172.21.15.96:0/2410804833 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7f3048038260 0x7f304803a770 unknown :-1 s=CLOSED pgs=3875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.840+0000 7f306f11c640 1 --2- 172.21.15.96:0/2410804833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f306809b360 0x7f306811f740 unknown :-1 s=CLOSED pgs=13020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.840+0000 7f306f11c640 1 -- 172.21.15.96:0/2410804833 >> 172.21.15.96:0/2410804833 conn(0x7f30680a42c0 msgr2=0x7f306809f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.840+0000 7f306f11c640 1 -- 172.21.15.96:0/2410804833 shutdown_connections 2024-03-21T06:39:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.840+0000 7f306f11c640 1 -- 172.21.15.96:0/2410804833 wait complete. 2024-03-21T06:39:24.850 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":1278,"stamp":"2024-03-21T06:39:22.866442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590373,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":144,"num_write_kb":1901,"num_scrub_errors":41,"num_shallow_scrub_errors":41,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":419430400,"kb_used":287728,"kb_used_data":14736,"kb_used_omap":0,"kb_used_meta":272960,"kb_avail":419142672,"statfs":{"total":429496729600,"available":429202096128,"internally_reserved":0,"allocated":15089664,"data_stored":6399673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":279511040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003534"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1277,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:03.313866+0000","last_change":"2024-03-21T06:39:03.327688+0000","last_active":"2024-03-21T06:39:03.313866+0000","last_peered":"2024-03-21T06:39:03.313866+0000","last_clean":"2024-03-21T06:39:03.313866+0000","last_became_active":"2024-03-21T06:25:58.480248+0000","last_became_peered":"2024-03-21T06:25:58.480248+0000","last_unstale":"2024-03-21T06:39:03.313866+0000","last_undegraded":"2024-03-21T06:39:03.313866+0000","last_fullsized":"2024-03-21T06:39:03.313866+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:03.327584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:03.327584+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1279,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:02.871844+0000","last_change":"2024-03-21T06:39:02.885741+0000","last_active":"2024-03-21T06:39:02.871844+0000","last_peered":"2024-03-21T06:39:02.871844+0000","last_clean":"2024-03-21T06:39:02.871844+0000","last_became_active":"2024-03-21T06:25:58.484415+0000","last_became_peered":"2024-03-21T06:25:58.484415+0000","last_unstale":"2024-03-21T06:39:02.871844+0000","last_undegraded":"2024-03-21T06:39:02.871844+0000","last_fullsized":"2024-03-21T06:39:02.871844+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:02.885648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:02.885648+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1287,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:03.759293+0000","last_change":"2024-03-21T06:39:03.775544+0000","last_active":"2024-03-21T06:39:03.759293+0000","last_peered":"2024-03-21T06:39:03.759293+0000","last_clean":"2024-03-21T06:39:03.759293+0000","last_became_active":"2024-03-21T06:25:58.487475+0000","last_became_peered":"2024-03-21T06:25:58.487475+0000","last_unstale":"2024-03-21T06:39:03.759293+0000","last_undegraded":"2024-03-21T06:39:03.759293+0000","last_fullsized":"2024-03-21T06:39:03.759293+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:03.775425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:03.775425+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1267,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:04.213051+0000","last_change":"2024-03-21T06:39:04.243386+0000","last_active":"2024-03-21T06:39:04.213051+0000","last_peered":"2024-03-21T06:39:04.213051+0000","last_clean":"2024-03-21T06:39:04.213051+0000","last_became_active":"2024-03-21T06:25:58.519468+0000","last_became_peered":"2024-03-21T06:25:58.519468+0000","last_unstale":"2024-03-21T06:39:04.213051+0000","last_undegraded":"2024-03-21T06:39:04.213051+0000","last_fullsized":"2024-03-21T06:39:04.213051+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:04.243269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:04.243269+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1277,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:03.313079+0000","last_change":"2024-03-21T06:39:03.332526+0000","last_active":"2024-03-21T06:39:03.313079+0000","last_peered":"2024-03-21T06:39:03.313079+0000","last_clean":"2024-03-21T06:39:03.313079+0000","last_became_active":"2024-03-21T06:25:58.527143+0000","last_became_peered":"2024-03-21T06:25:58.527143+0000","last_unstale":"2024-03-21T06:39:03.313079+0000","last_undegraded":"2024-03-21T06:39:03.313079+0000","last_fullsized":"2024-03-21T06:39:03.313079+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:03.332427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:03.332427+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1279,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:02.869724+0000","last_change":"2024-03-21T06:39:02.884567+0000","last_active":"2024-03-21T06:39:02.869724+0000","last_peered":"2024-03-21T06:39:02.869724+0000","last_clean":"2024-03-21T06:39:02.869724+0000","last_became_active":"2024-03-21T06:25:58.494870+0000","last_became_peered":"2024-03-21T06:25:58.494870+0000","last_unstale":"2024-03-21T06:39:02.869724+0000","last_undegraded":"2024-03-21T06:39:02.869724+0000","last_fullsized":"2024-03-21T06:39:02.869724+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:02.884470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:02.884470+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1279,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:02.869367+0000","last_change":"2024-03-21T06:39:02.892945+0000","last_active":"2024-03-21T06:39:02.869367+0000","last_peered":"2024-03-21T06:39:02.869367+0000","last_clean":"2024-03-21T06:39:02.869367+0000","last_became_active":"2024-03-21T06:25:58.488027+0000","last_became_peered":"2024-03-21T06:25:58.488027+0000","last_unstale":"2024-03-21T06:39:02.869367+0000","last_undegraded":"2024-03-21T06:39:02.869367+0000","last_fullsized":"2024-03-21T06:39:02.869367+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:02.892849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:02.892849+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1279,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:02.868178+0000","last_change":"2024-03-21T06:39:02.892247+0000","last_active":"2024-03-21T06:39:02.868178+0000","last_peered":"2024-03-21T06:39:02.868178+0000","last_clean":"2024-03-21T06:39:02.868178+0000","last_became_active":"2024-03-21T06:25:58.498450+0000","last_became_peered":"2024-03-21T06:25:58.498450+0000","last_unstale":"2024-03-21T06:39:02.868178+0000","last_undegraded":"2024-03-21T06:39:02.868178+0000","last_fullsized":"2024-03-21T06:39:02.868178+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:02.892144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:02.892144+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1267,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:04.211997+0000","last_change":"2024-03-21T06:39:04.242317+0000","last_active":"2024-03-21T06:39:04.211997+0000","last_peered":"2024-03-21T06:39:04.211997+0000","last_clean":"2024-03-21T06:39:04.211997+0000","last_became_active":"2024-03-21T06:25:58.503376+0000","last_became_peered":"2024-03-21T06:25:58.503376+0000","last_unstale":"2024-03-21T06:39:04.211997+0000","last_undegraded":"2024-03-21T06:39:04.211997+0000","last_fullsized":"2024-03-21T06:39:04.211997+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:04.242131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:04.242131+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1287,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:03.758227+0000","last_change":"2024-03-21T06:39:03.774332+0000","last_active":"2024-03-21T06:39:03.758227+0000","last_peered":"2024-03-21T06:39:03.758227+0000","last_clean":"2024-03-21T06:39:03.758227+0000","last_became_active":"2024-03-21T06:25:58.497498+0000","last_became_peered":"2024-03-21T06:25:58.497498+0000","last_unstale":"2024-03-21T06:39:03.758227+0000","last_undegraded":"2024-03-21T06:39:03.758227+0000","last_fullsized":"2024-03-21T06:39:03.758227+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:03.774209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:03.774209+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1267,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:04.210968+0000","last_change":"2024-03-21T06:39:04.239825+0000","last_active":"2024-03-21T06:39:04.210968+0000","last_peered":"2024-03-21T06:39:04.210968+0000","last_clean":"2024-03-21T06:39:04.210968+0000","last_became_active":"2024-03-21T06:25:58.498895+0000","last_became_peered":"2024-03-21T06:25:58.498895+0000","last_unstale":"2024-03-21T06:39:04.210968+0000","last_undegraded":"2024-03-21T06:39:04.210968+0000","last_fullsized":"2024-03-21T06:39:04.210968+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:04.239698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:04.239698+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1279,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:02.867568+0000","last_change":"2024-03-21T06:39:02.891415+0000","last_active":"2024-03-21T06:39:02.867568+0000","last_peered":"2024-03-21T06:39:02.867568+0000","last_clean":"2024-03-21T06:39:02.867568+0000","last_became_active":"2024-03-21T06:25:58.486238+0000","last_became_peered":"2024-03-21T06:25:58.486238+0000","last_unstale":"2024-03-21T06:39:02.867568+0000","last_undegraded":"2024-03-21T06:39:02.867568+0000","last_fullsized":"2024-03-21T06:39:02.867568+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:02.891302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:02.891302+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1267,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:04.209008+0000","last_change":"2024-03-21T06:39:04.235812+0000","last_active":"2024-03-21T06:39:04.209008+0000","last_peered":"2024-03-21T06:39:04.209008+0000","last_clean":"2024-03-21T06:39:04.209008+0000","last_became_active":"2024-03-21T06:25:58.513806+0000","last_became_peered":"2024-03-21T06:25:58.513806+0000","last_unstale":"2024-03-21T06:39:04.209008+0000","last_undegraded":"2024-03-21T06:39:04.209008+0000","last_fullsized":"2024-03-21T06:39:04.209008+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:04.235695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:04.235695+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1267,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:04.207816+0000","last_change":"2024-03-21T06:39:04.237458+0000","last_active":"2024-03-21T06:39:04.207816+0000","last_peered":"2024-03-21T06:39:04.207816+0000","last_clean":"2024-03-21T06:39:04.207816+0000","last_became_active":"2024-03-21T06:25:58.537009+0000","last_became_peered":"2024-03-21T06:25:58.537009+0000","last_unstale":"2024-03-21T06:39:04.207816+0000","last_undegraded":"2024-03-21T06:39:04.207816+0000","last_fullsized":"2024-03-21T06:39:04.207816+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:04.237341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:04.237341+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1279,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:02.866223+0000","last_change":"2024-03-21T06:39:02.889918+0000","last_active":"2024-03-21T06:39:02.866223+0000","last_peered":"2024-03-21T06:39:02.866223+0000","last_clean":"2024-03-21T06:39:02.866223+0000","last_became_active":"2024-03-21T06:25:58.510651+0000","last_became_peered":"2024-03-21T06:25:58.510651+0000","last_unstale":"2024-03-21T06:39:02.866223+0000","last_undegraded":"2024-03-21T06:39:02.866223+0000","last_fullsized":"2024-03-21T06:39:02.866223+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:02.889820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:02.889820+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1279,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:02.865301+0000","last_change":"2024-03-21T06:39:02.888079+0000","last_active":"2024-03-21T06:39:02.865301+0000","last_peered":"2024-03-21T06:39:02.865301+0000","last_clean":"2024-03-21T06:39:02.865301+0000","last_became_active":"2024-03-21T06:25:58.548956+0000","last_became_peered":"2024-03-21T06:25:58.548956+0000","last_unstale":"2024-03-21T06:39:02.865301+0000","last_undegraded":"2024-03-21T06:39:02.865301+0000","last_fullsized":"2024-03-21T06:39:02.865301+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:02.887970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:02.887970+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1279,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:02.863321+0000","last_change":"2024-03-21T06:39:02.890618+0000","last_active":"2024-03-21T06:39:02.863321+0000","last_peered":"2024-03-21T06:39:02.863321+0000","last_clean":"2024-03-21T06:39:02.863321+0000","last_became_active":"2024-03-21T06:25:58.482358+0000","last_became_peered":"2024-03-21T06:25:58.482358+0000","last_unstale":"2024-03-21T06:39:02.863321+0000","last_undegraded":"2024-03-21T06:39:02.863321+0000","last_fullsized":"2024-03-21T06:39:02.863321+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:02.890522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:02.890522+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1267,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:04.207403+0000","last_change":"2024-03-21T06:39:04.240748+0000","last_active":"2024-03-21T06:39:04.207403+0000","last_peered":"2024-03-21T06:39:04.207403+0000","last_clean":"2024-03-21T06:39:04.207403+0000","last_became_active":"2024-03-21T06:25:58.521822+0000","last_became_peered":"2024-03-21T06:25:58.521822+0000","last_unstale":"2024-03-21T06:39:04.207403+0000","last_undegraded":"2024-03-21T06:39:04.207403+0000","last_fullsized":"2024-03-21T06:39:04.207403+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:04.240633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:04.240633+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1287,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:03.756868+0000","last_change":"2024-03-21T06:39:03.773638+0000","last_active":"2024-03-21T06:39:03.756868+0000","last_peered":"2024-03-21T06:39:03.756868+0000","last_clean":"2024-03-21T06:39:03.756868+0000","last_became_active":"2024-03-21T06:25:58.484498+0000","last_became_peered":"2024-03-21T06:25:58.484498+0000","last_unstale":"2024-03-21T06:39:03.756868+0000","last_undegraded":"2024-03-21T06:39:03.756868+0000","last_fullsized":"2024-03-21T06:39:03.756868+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:03.773517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:03.773517+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1277,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:03.311837+0000","last_change":"2024-03-21T06:39:03.331574+0000","last_active":"2024-03-21T06:39:03.311837+0000","last_peered":"2024-03-21T06:39:03.311837+0000","last_clean":"2024-03-21T06:39:03.311837+0000","last_became_active":"2024-03-21T06:25:58.474376+0000","last_became_peered":"2024-03-21T06:25:58.474376+0000","last_unstale":"2024-03-21T06:39:03.311837+0000","last_undegraded":"2024-03-21T06:39:03.311837+0000","last_fullsized":"2024-03-21T06:39:03.311837+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:03.331470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:03.331470+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1277,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:03.311187+0000","last_change":"2024-03-21T06:39:03.329744+0000","last_active":"2024-03-21T06:39:03.311187+0000","last_peered":"2024-03-21T06:39:03.311187+0000","last_clean":"2024-03-21T06:39:03.311187+0000","last_became_active":"2024-03-21T06:25:58.477107+0000","last_became_peered":"2024-03-21T06:25:58.477107+0000","last_unstale":"2024-03-21T06:39:03.311187+0000","last_undegraded":"2024-03-21T06:39:03.311187+0000","last_fullsized":"2024-03-21T06:39:03.311187+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:03.329645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:03.329645+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1277,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:03.310582+0000","last_change":"2024-03-21T06:39:03.330767+0000","last_active":"2024-03-21T06:39:03.310582+0000","last_peered":"2024-03-21T06:39:03.310582+0000","last_clean":"2024-03-21T06:39:03.310582+0000","last_became_active":"2024-03-21T06:25:58.490894+0000","last_became_peered":"2024-03-21T06:25:58.490894+0000","last_unstale":"2024-03-21T06:39:03.310582+0000","last_undegraded":"2024-03-21T06:39:03.310582+0000","last_fullsized":"2024-03-21T06:39:03.310582+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:03.330669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:03.330669+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1287,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:03.756266+0000","last_change":"2024-03-21T06:39:03.771536+0000","last_active":"2024-03-21T06:39:03.756266+0000","last_peered":"2024-03-21T06:39:03.756266+0000","last_clean":"2024-03-21T06:39:03.756266+0000","last_became_active":"2024-03-21T06:25:58.505636+0000","last_became_peered":"2024-03-21T06:25:58.505636+0000","last_unstale":"2024-03-21T06:39:03.756266+0000","last_undegraded":"2024-03-21T06:39:03.756266+0000","last_fullsized":"2024-03-21T06:39:03.756266+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:03.771414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:03.771414+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1267,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:04.206088+0000","last_change":"2024-03-21T06:39:04.238622+0000","last_active":"2024-03-21T06:39:04.206088+0000","last_peered":"2024-03-21T06:39:04.206088+0000","last_clean":"2024-03-21T06:39:04.206088+0000","last_became_active":"2024-03-21T06:25:58.501608+0000","last_became_peered":"2024-03-21T06:25:58.501608+0000","last_unstale":"2024-03-21T06:39:04.206088+0000","last_undegraded":"2024-03-21T06:39:04.206088+0000","last_fullsized":"2024-03-21T06:39:04.206088+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:04.238504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:04.238504+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1267,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:04.204535+0000","last_change":"2024-03-21T06:39:04.234196+0000","last_active":"2024-03-21T06:39:04.204535+0000","last_peered":"2024-03-21T06:39:04.204535+0000","last_clean":"2024-03-21T06:39:04.204535+0000","last_became_active":"2024-03-21T06:25:58.508192+0000","last_became_peered":"2024-03-21T06:25:58.508192+0000","last_unstale":"2024-03-21T06:39:04.204535+0000","last_undegraded":"2024-03-21T06:39:04.204535+0000","last_fullsized":"2024-03-21T06:39:04.204535+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:04.234055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:04.234055+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1279,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:02.863003+0000","last_change":"2024-03-21T06:39:02.889052+0000","last_active":"2024-03-21T06:39:02.863003+0000","last_peered":"2024-03-21T06:39:02.863003+0000","last_clean":"2024-03-21T06:39:02.863003+0000","last_became_active":"2024-03-21T06:25:58.491525+0000","last_became_peered":"2024-03-21T06:25:58.491525+0000","last_unstale":"2024-03-21T06:39:02.863003+0000","last_undegraded":"2024-03-21T06:39:02.863003+0000","last_fullsized":"2024-03-21T06:39:02.863003+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:02.888939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:02.888939+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1267,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:04.203778+0000","last_change":"2024-03-21T06:39:04.227736+0000","last_active":"2024-03-21T06:39:04.203778+0000","last_peered":"2024-03-21T06:39:04.203778+0000","last_clean":"2024-03-21T06:39:04.203778+0000","last_became_active":"2024-03-21T06:25:58.506657+0000","last_became_peered":"2024-03-21T06:25:58.506657+0000","last_unstale":"2024-03-21T06:39:04.203778+0000","last_undegraded":"2024-03-21T06:39:04.203778+0000","last_fullsized":"2024-03-21T06:39:04.203778+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:04.227612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:04.227612+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1279,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:02.861824+0000","last_change":"2024-03-21T06:39:02.883069+0000","last_active":"2024-03-21T06:39:02.861824+0000","last_peered":"2024-03-21T06:39:02.861824+0000","last_clean":"2024-03-21T06:39:02.861824+0000","last_became_active":"2024-03-21T06:25:58.517030+0000","last_became_peered":"2024-03-21T06:25:58.517030+0000","last_unstale":"2024-03-21T06:39:02.861824+0000","last_undegraded":"2024-03-21T06:39:02.861824+0000","last_fullsized":"2024-03-21T06:39:02.861824+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:02.882945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:02.882945+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1279,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:02.861190+0000","last_change":"2024-03-21T06:39:02.887597+0000","last_active":"2024-03-21T06:39:02.861190+0000","last_peered":"2024-03-21T06:39:02.861190+0000","last_clean":"2024-03-21T06:39:02.861190+0000","last_became_active":"2024-03-21T06:25:58.528601+0000","last_became_peered":"2024-03-21T06:25:58.528601+0000","last_unstale":"2024-03-21T06:39:02.861190+0000","last_undegraded":"2024-03-21T06:39:02.861190+0000","last_fullsized":"2024-03-21T06:39:02.861190+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:02.887500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:02.887500+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1279,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:02.860632+0000","last_change":"2024-03-21T06:39:02.885300+0000","last_active":"2024-03-21T06:39:02.860632+0000","last_peered":"2024-03-21T06:39:02.860632+0000","last_clean":"2024-03-21T06:39:02.860632+0000","last_became_active":"2024-03-21T06:25:58.503544+0000","last_became_peered":"2024-03-21T06:25:58.503544+0000","last_unstale":"2024-03-21T06:39:02.860632+0000","last_undegraded":"2024-03-21T06:39:02.860632+0000","last_fullsized":"2024-03-21T06:39:02.860632+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:02.885203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:02.885203+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"236'7","reported_seq":1290,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:03.308949+0000","last_change":"2024-03-21T06:39:03.333231+0000","last_active":"2024-03-21T06:39:03.308949+0000","last_peered":"2024-03-21T06:39:03.308949+0000","last_clean":"2024-03-21T06:39:03.308949+0000","last_became_active":"2024-03-21T06:25:58.505585+0000","last_became_peered":"2024-03-21T06:25:58.505585+0000","last_unstale":"2024-03-21T06:39:03.308949+0000","last_undegraded":"2024-03-21T06:39:03.308949+0000","last_fullsized":"2024-03-21T06:39:03.308949+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'7","last_scrub_stamp":"2024-03-21T06:39:03.333111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:03.333111+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1287,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:03.755250+0000","last_change":"2024-03-21T06:39:03.770123+0000","last_active":"2024-03-21T06:39:03.755250+0000","last_peered":"2024-03-21T06:39:03.755250+0000","last_clean":"2024-03-21T06:39:03.755250+0000","last_became_active":"2024-03-21T06:25:58.481996+0000","last_became_peered":"2024-03-21T06:25:58.481996+0000","last_unstale":"2024-03-21T06:39:03.755250+0000","last_undegraded":"2024-03-21T06:39:03.755250+0000","last_fullsized":"2024-03-21T06:39:03.755250+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:03.769988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:03.769988+0000","last_clean_scrub_stamp":"2024-03-21T06:25:57.372019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1876,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:03.308050+0000","last_change":"2024-03-21T06:39:03.324649+0000","last_active":"2024-03-21T06:39:03.308050+0000","last_peered":"2024-03-21T06:39:03.308050+0000","last_clean":"2024-03-21T06:39:03.308050+0000","last_became_active":"2024-03-21T06:20:23.721083+0000","last_became_peered":"2024-03-21T06:20:23.721083+0000","last_unstale":"2024-03-21T06:39:03.308050+0000","last_undegraded":"2024-03-21T06:39:03.308050+0000","last_fullsized":"2024-03-21T06:39:03.308050+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:03.324535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:03.324535+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1886,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:03.755260+0000","last_change":"2024-03-21T06:39:03.765868+0000","last_active":"2024-03-21T06:39:03.755260+0000","last_peered":"2024-03-21T06:39:03.755260+0000","last_clean":"2024-03-21T06:39:03.755260+0000","last_became_active":"2024-03-21T06:20:23.717296+0000","last_became_peered":"2024-03-21T06:20:23.717296+0000","last_unstale":"2024-03-21T06:39:03.755260+0000","last_undegraded":"2024-03-21T06:39:03.755260+0000","last_fullsized":"2024-03-21T06:39:03.755260+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:03.765686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:03.765686+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1870,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:04.203594+0000","last_change":"2024-03-21T06:39:04.226222+0000","last_active":"2024-03-21T06:39:04.203594+0000","last_peered":"2024-03-21T06:39:04.203594+0000","last_clean":"2024-03-21T06:39:04.203594+0000","last_became_active":"2024-03-21T06:20:23.721198+0000","last_became_peered":"2024-03-21T06:20:23.721198+0000","last_unstale":"2024-03-21T06:39:04.203594+0000","last_undegraded":"2024-03-21T06:39:04.203594+0000","last_fullsized":"2024-03-21T06:39:04.203594+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:04.225916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:04.225916+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1886,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:03.754140+0000","last_change":"2024-03-21T06:39:03.772659+0000","last_active":"2024-03-21T06:39:03.754140+0000","last_peered":"2024-03-21T06:39:03.754140+0000","last_clean":"2024-03-21T06:39:03.754140+0000","last_became_active":"2024-03-21T06:20:23.714274+0000","last_became_peered":"2024-03-21T06:20:23.714274+0000","last_unstale":"2024-03-21T06:39:03.754140+0000","last_undegraded":"2024-03-21T06:39:03.754140+0000","last_fullsized":"2024-03-21T06:39:03.754140+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:03.772529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:03.772529+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1872,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:02.860646+0000","last_change":"2024-03-21T06:39:02.867991+0000","last_active":"2024-03-21T06:39:02.860646+0000","last_peered":"2024-03-21T06:39:02.860646+0000","last_clean":"2024-03-21T06:39:02.860646+0000","last_became_active":"2024-03-21T06:20:23.714766+0000","last_became_peered":"2024-03-21T06:20:23.714766+0000","last_unstale":"2024-03-21T06:39:02.860646+0000","last_undegraded":"2024-03-21T06:39:02.860646+0000","last_fullsized":"2024-03-21T06:39:02.860646+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:02.867861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:02.867861+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1876,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:03.307216+0000","last_change":"2024-03-21T06:39:03.325364+0000","last_active":"2024-03-21T06:39:03.307216+0000","last_peered":"2024-03-21T06:39:03.307216+0000","last_clean":"2024-03-21T06:39:03.307216+0000","last_became_active":"2024-03-21T06:20:23.712781+0000","last_became_peered":"2024-03-21T06:20:23.712781+0000","last_unstale":"2024-03-21T06:39:03.307216+0000","last_undegraded":"2024-03-21T06:39:03.307216+0000","last_fullsized":"2024-03-21T06:39:03.307216+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:03.325271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:03.325271+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1872,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:02.860631+0000","last_change":"2024-03-21T06:39:02.877718+0000","last_active":"2024-03-21T06:39:02.860631+0000","last_peered":"2024-03-21T06:39:02.860631+0000","last_clean":"2024-03-21T06:39:02.860631+0000","last_became_active":"2024-03-21T06:20:23.720894+0000","last_became_peered":"2024-03-21T06:20:23.720894+0000","last_unstale":"2024-03-21T06:39:02.860631+0000","last_undegraded":"2024-03-21T06:39:02.860631+0000","last_fullsized":"2024-03-21T06:39:02.860631+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:02.877603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:02.877603+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1876,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:03.308203+0000","last_change":"2024-03-21T06:39:03.327587+0000","last_active":"2024-03-21T06:39:03.308203+0000","last_peered":"2024-03-21T06:39:03.308203+0000","last_clean":"2024-03-21T06:39:03.308203+0000","last_became_active":"2024-03-21T06:20:23.716768+0000","last_became_peered":"2024-03-21T06:20:23.716768+0000","last_unstale":"2024-03-21T06:39:03.308203+0000","last_undegraded":"2024-03-21T06:39:03.308203+0000","last_fullsized":"2024-03-21T06:39:03.308203+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T06:39:03.327479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:03.327479+0000","last_clean_scrub_stamp":"2024-03-21T06:20:22.674032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":2003,"reported_epoch":713,"state":"active+clean+inconsistent","last_fresh":"2024-03-21T06:39:04.202337+0000","last_change":"2024-03-21T06:39:04.244822+0000","last_active":"2024-03-21T06:39:04.202337+0000","last_peered":"2024-03-21T06:39:04.202337+0000","last_clean":"2024-03-21T06:39:04.202337+0000","last_became_active":"2024-03-21T06:20:21.690580+0000","last_became_peered":"2024-03-21T06:20:21.690580+0000","last_unstale":"2024-03-21T06:39:04.202337+0000","last_undegraded":"2024-03-21T06:39:04.202337+0000","last_fullsized":"2024-03-21T06:39:04.202337+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"11'97","last_scrub_stamp":"2024-03-21T06:39:04.244665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T06:39:04.244665+0000","last_clean_scrub_stamp":"2024-03-21T06:20:20.658895+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":32,"num_shallow_scrub_errors":32,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":8,"num_shallow_scrub_errors":8,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":1,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706803,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":71156,"kb_used_data":3948,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":104786444,"statfs":{"total":107374182400,"available":107301318656,"internally_reserved":0,"allocated":4042752,"data_stored":1897125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706803,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":70624,"kb_used_data":3416,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":104786976,"statfs":{"total":107374182400,"available":107301863424,"internally_reserved":0,"allocated":3497984,"data_stored":1304628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706803,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":70608,"kb_used_data":3400,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":104786992,"statfs":{"total":107374182400,"available":107301879808,"internally_reserved":0,"allocated":3481600,"data_stored":1303794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706803,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":75340,"kb_used_data":3972,"kb_used_omap":0,"kb_used_meta":71360,"kb_avail":104782260,"statfs":{"total":107374182400,"available":107297034240,"internally_reserved":0,"allocated":4067328,"data_stored":1894126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":73072640},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-03-21T06:39:24.853 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-21T06:39:24.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.952+0000 7fab102dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:24.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.959+0000 7fab102dc640 1 Processor -- start 2024-03-21T06:39:24.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.959+0000 7fab102dc640 1 -- start start 2024-03-21T06:39:24.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.960+0000 7fab102dc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fab08130b70 0x7fab08135990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:24.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.960+0000 7fab102dc640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab08099dc0 con 0x7fab0809b5d0 2024-03-21T06:39:24.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.960+0000 7fab102dc640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab0809bae0 con 0x7fab08130b70 2024-03-21T06:39:24.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.960+0000 7fab0e078640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fab0809b5d0 0x7fab08130460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33884/0 (socket says 172.21.15.96:33884) 2024-03-21T06:39:24.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.960+0000 7fab0e078640 1 -- 172.21.15.96:0/1465468264 learned_addr learned my addr 172.21.15.96:0/1465468264 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-03-21T06:39:24.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.960+0000 7fab0d877640 1 --2- 172.21.15.96:0/1465468264 >> v2:172.21.15.96:3300/0 conn(0x7fab08130b70 0x7fab08135990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:24.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.961+0000 7fab0d076640 1 -- 172.21.15.96:0/1465468264 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922285992 0 0) 0x7fab08099dc0 con 0x7fab0809b5d0 2024-03-21T06:39:24.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.961+0000 7fab0d076640 1 -- 172.21.15.96:0/1465468264 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaf40034f0 con 0x7fab0809b5d0 2024-03-21T06:39:24.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.961+0000 7fab0d877640 1 -- 172.21.15.96:0/1465468264 >> v1:172.21.15.96:6789/0 conn(0x7fab0809b5d0 legacy=0x7fab08130460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:24.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.961+0000 7fab0d877640 1 -- 172.21.15.96:0/1465468264 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0809c0f0 con 0x7fab08130b70 2024-03-21T06:39:24.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.961+0000 7fab0d877640 1 --2- 172.21.15.96:0/1465468264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab08130b70 0x7fab08135990 crc :-1 s=READY pgs=13022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b632a6b6978a41 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:24.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.961+0000 7fab0d076640 1 -- 172.21.15.96:0/1465468264 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7faaf800ac70 con 0x7fab08130b70 2024-03-21T06:39:24.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.961+0000 7fab0d076640 1 -- 172.21.15.96:0/1465468264 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7faaf800add0 con 0x7fab08130b70 2024-03-21T06:39:24.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.961+0000 7fab0d076640 1 -- 172.21.15.96:0/1465468264 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7faaf800f470 con 0x7fab08130b70 2024-03-21T06:39:24.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.962+0000 7fab102dc640 1 -- 172.21.15.96:0/1465468264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab08130b70 msgr2=0x7fab08135990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:24.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.962+0000 7fab102dc640 1 --2- 172.21.15.96:0/1465468264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab08130b70 0x7fab08135990 crc :-1 s=READY pgs=13022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:24.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.962+0000 7fab102dc640 1 -- 172.21.15.96:0/1465468264 shutdown_connections 2024-03-21T06:39:24.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.962+0000 7fab102dc640 1 --2- 172.21.15.96:0/1465468264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab08130b70 0x7fab08135990 unknown :-1 s=CLOSED pgs=13022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:24.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.962+0000 7fab102dc640 1 -- 172.21.15.96:0/1465468264 >> 172.21.15.96:0/1465468264 conn(0x7fab0809fdb0 msgr2=0x7fab080a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:24.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.962+0000 7fab102dc640 1 -- 172.21.15.96:0/1465468264 shutdown_connections 2024-03-21T06:39:24.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.962+0000 7fab102dc640 1 -- 172.21.15.96:0/1465468264 wait complete. 2024-03-21T06:39:24.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.962+0000 7fab102dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-03-21T06:39:24.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.963+0000 7fab102dc640 1 Processor -- start 2024-03-21T06:39:24.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.963+0000 7fab102dc640 1 -- start start 2024-03-21T06:39:24.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.964+0000 7fab102dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab0809b5d0 0x7fab08070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:24.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.964+0000 7fab102dc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab08073920 con 0x7fab0809b5d0 2024-03-21T06:39:24.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.964+0000 7fab0e078640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab0809b5d0 0x7fab08070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:24.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.964+0000 7fab0e078640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab0809b5d0 0x7fab08070a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43810/0 (socket says 172.21.15.96:43810) 2024-03-21T06:39:24.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.964+0000 7fab0e078640 1 -- 172.21.15.96:0/1993263768 learned_addr learned my addr 172.21.15.96:0/1993263768 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-21T06:39:24.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.964+0000 7fab0e078640 1 -- 172.21.15.96:0/1993263768 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab08128780 con 0x7fab0809b5d0 2024-03-21T06:39:24.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.964+0000 7fab0e078640 1 --2- 172.21.15.96:0/1993263768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab0809b5d0 0x7fab08070a50 crc :-1 s=READY pgs=13023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:24.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.964+0000 7faafeffd640 1 -- 172.21.15.96:0/1993263768 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fab040059d0 con 0x7fab0809b5d0 2024-03-21T06:39:24.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.964+0000 7faafeffd640 1 -- 172.21.15.96:0/1993263768 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (crc 0 0 0) 0x7fab04005b30 con 0x7fab0809b5d0 2024-03-21T06:39:24.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.964+0000 7faafeffd640 1 -- 172.21.15.96:0/1993263768 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (crc 0 0 0) 0x7fab04005dd0 con 0x7fab0809b5d0 2024-03-21T06:39:24.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.964+0000 7fab102dc640 1 -- 172.21.15.96:0/1993263768 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab08072680 con 0x7fab0809b5d0 2024-03-21T06:39:24.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.964+0000 7fab102dc640 1 -- 172.21.15.96:0/1993263768 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab08128350 con 0x7fab0809b5d0 2024-03-21T06:39:24.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.965+0000 7faafeffd640 1 -- 172.21.15.96:0/1993263768 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 45382+0+0 (crc 0 0 0) 0x7fab040070b0 con 0x7fab0809b5d0 2024-03-21T06:39:24.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.965+0000 7faafeffd640 1 --2- 172.21.15.96:0/1993263768 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7faaec038260 0x7faaec03a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T06:39:24.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.965+0000 7faafeffd640 1 -- 172.21.15.96:0/1993263768 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(713..713 src has 402..713) v4 ==== 3403+0+0 (crc 0 0 0) 0x7fab04047b10 con 0x7fab0809b5d0 2024-03-21T06:39:24.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.965+0000 7fab102dc640 1 -- 172.21.15.96:0/1993263768 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab08062f10 con 0x7fab0809b5d0 2024-03-21T06:39:24.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.965+0000 7fab0d877640 1 --2- 172.21.15.96:0/1993263768 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7faaec038260 0x7faaec03a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T06:39:24.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.966+0000 7fab0d877640 1 --2- 172.21.15.96:0/1993263768 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7faaec038260 0x7faaec03a770 crc :-1 s=READY pgs=3876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T06:39:24.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:24.970+0000 7faafeffd640 1 -- 172.21.15.96:0/1993263768 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185025 (crc 0 0 0) 0x7fab0400da20 con 0x7fab0809b5d0 2024-03-21T06:39:25.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:25.144+0000 7fab102dc640 1 -- 172.21.15.96:0/1993263768 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fab08072420 con 0x7fab0809b5d0 2024-03-21T06:39:25.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:25.147+0000 7faafeffd640 1 -- 172.21.15.96:0/1993263768 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v2)=0 v2) v1 ==== 123+0+0 (crc 0 0 0) 0x7fab0400d610 con 0x7fab0809b5d0 2024-03-21T06:39:25.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:25.148+0000 7faafeffd640 1 -- 172.21.15.96:0/1993263768 <== mon.0 v2:172.21.15.96:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (crc 0 0 0) 0x7fab0400da20 con 0x7fab0809b5d0 2024-03-21T06:39:25.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:25.149+0000 7fab102dc640 1 -- 172.21.15.96:0/1993263768 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7faaec038260 msgr2=0x7faaec03a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:25.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:25.149+0000 7fab102dc640 1 --2- 172.21.15.96:0/1993263768 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7faaec038260 0x7faaec03a770 crc :-1 s=READY pgs=3876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:25.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:25.149+0000 7fab102dc640 1 -- 172.21.15.96:0/1993263768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab0809b5d0 msgr2=0x7fab08070a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T06:39:25.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:25.149+0000 7fab102dc640 1 --2- 172.21.15.96:0/1993263768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab0809b5d0 0x7fab08070a50 crc :-1 s=READY pgs=13023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:25.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:25.149+0000 7fab102dc640 1 -- 172.21.15.96:0/1993263768 shutdown_connections 2024-03-21T06:39:25.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:25.149+0000 7fab102dc640 1 --2- 172.21.15.96:0/1993263768 >> [v2:172.21.15.184:6800/2376803682,v1:172.21.15.184:6801/2376803682] conn(0x7faaec038260 0x7faaec03a770 unknown :-1 s=CLOSED pgs=3876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:25.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:25.149+0000 7fab102dc640 1 --2- 172.21.15.96:0/1993263768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab0809b5d0 0x7fab08070a50 unknown :-1 s=CLOSED pgs=13023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T06:39:25.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:25.149+0000 7fab102dc640 1 -- 172.21.15.96:0/1993263768 >> 172.21.15.96:0/1993263768 conn(0x7fab0809fdb0 msgr2=0x7fab080a0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T06:39:25.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:25.149+0000 7fab102dc640 1 -- 172.21.15.96:0/1993263768 shutdown_connections 2024-03-21T06:39:25.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-21T06:39:25.149+0000 7fab102dc640 1 -- 172.21.15.96:0/1993263768 wait complete. 2024-03-21T06:39:25.160 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-21T06:39:25.160 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-21T06:39:25.160 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-21T06:39:25.161 INFO:teuthology.orchestra.run:waiting for 300 2024-03-21T06:39:25.259 INFO:tasks.ceph.osd.0:Stopped 2024-03-21T06:39:25.260 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-21T06:39:25.260 INFO:teuthology.orchestra.run:waiting for 300 2024-03-21T06:39:25.354 INFO:tasks.ceph.osd.1:Stopped 2024-03-21T06:39:25.354 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-21T06:39:25.354 INFO:teuthology.orchestra.run:waiting for 300 2024-03-21T06:39:27.502 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-03-21T06:39:31.457 INFO:tasks.ceph.osd.2:Stopped 2024-03-21T06:39:31.458 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-21T06:39:31.458 INFO:teuthology.orchestra.run:waiting for 300 2024-03-21T06:39:32.704 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-03-21T06:39:32.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-03-21T06:39:37.559 INFO:tasks.ceph.osd.3:Stopped 2024-03-21T06:39:37.560 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-21T06:39:37.560 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-21T06:39:37.560 INFO:teuthology.orchestra.run:waiting for 300 2024-03-21T06:39:37.591 INFO:tasks.ceph.mgr.x:Stopped 2024-03-21T06:39:37.591 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-21T06:39:37.592 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-21T06:39:37.592 INFO:teuthology.orchestra.run:waiting for 300 2024-03-21T06:39:37.609 INFO:tasks.ceph.mon.a:Stopped 2024-03-21T06:39:37.609 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-21T06:39:37.609 DEBUG:teuthology.orchestra.run.smithi096:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-21T06:39:37.699 INFO:teuthology.orchestra.run.smithi096.stdout:2024-03-21T06:39:05.491020+0000 mon.a (mon.0) 1342 : cluster [ERR] Health check failed: 23 scrub errors (OSD_SCRUB_ERRORS) 2024-03-21T06:39:37.700 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-03-21T06:39:37.700 DEBUG:teuthology.orchestra.run.smithi096:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-21T06:39:37.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-03-21T06:39:37.733 DEBUG:teuthology.orchestra.run.smithi096:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-21T06:39:37.804 INFO:teuthology.orchestra.run.smithi096.stdout:2024-03-21T06:39:05.491020+0000 mon.a (mon.0) 1342 : cluster [ERR] Health check failed: 23 scrub errors (OSD_SCRUB_ERRORS) 2024-03-21T06:39:37.805 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi096.front.sepia.ceph.com 2024-03-21T06:39:37.806 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-21T06:39:37.927 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi096.front.sepia.ceph.com 2024-03-21T06:39:37.927 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-21T06:39:37.999 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi184.front.sepia.ceph.com 2024-03-21T06:39:38.000 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-21T06:39:38.269 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi184.front.sepia.ceph.com 2024-03-21T06:39:38.269 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-21T06:39:38.326 INFO:tasks.ceph:Archiving mon data... 2024-03-21T06:39:38.328 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi/7614674/data/mon.a.tgz 2024-03-21T06:39:38.328 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-03-21T06:39:38.343 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.mUuzXo4fLC 2024-03-21T06:39:38.343 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.mUuzXo4fLC 2024-03-21T06:39:38.487 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.mUuzXo4fLC 2024-03-21T06:39:38.572 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.mUuzXo4fLC is 308KB 2024-03-21T06:39:38.668 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.mUuzXo4fLC 2024-03-21T06:39:38.682 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-21T06:39:38.682 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-21T06:39:38.724 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-21T06:39:38.803 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-21T06:39:38.820 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-21T06:39:38.820 INFO:tasks.ceph:Archiving crash dumps... 2024-03-21T06:39:38.823 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/crash to /home/teuthworker/archive/yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi/7614674/remote/smithi096/crash 2024-03-21T06:39:38.824 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-21T06:39:38.870 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/crash to /home/teuthworker/archive/yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi/7614674/remote/smithi184/crash 2024-03-21T06:39:38.871 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-21T06:39:38.916 INFO:tasks.ceph:Compressing logs... 2024-03-21T06:39:38.916 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T06:39:38.919 DEBUG:teuthology.orchestra.run.smithi184:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T06:39:38.943 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31874.log 2024-03-21T06:39:38.943 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95985.log 2024-03-21T06:39:38.943 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-21T06:39:38.956 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30299.log 2024-03-21T06:39:38.956 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30379.log 2024-03-21T06:39:38.956 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30339.log 2024-03-21T06:39:38.956 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.30299.log: /var/log/ceph/ceph-client.admin.30379.log: gzip -5 84.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.30299.log.gz 2024-03-21T06:39:38.956 INFO:teuthology.orchestra.run.smithi184.stderr: /var/log/ceph/ceph-osd.3.log 2024-03-21T06:39:38.956 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.30339.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.30379.log.gz 2024-03-21T06:39:38.956 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-21T06:39:38.956 INFO:teuthology.orchestra.run.smithi184.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.30339.log.gz 2024-03-21T06:39:38.956 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30259.log 2024-03-21T06:39:38.957 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-21T06:39:38.957 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.30259.log: /var/log/ceph/ceph-osd.2.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.30259.log.gz 2024-03-21T06:39:39.023 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.31874.log: /var/log/ceph/ceph-client.admin.95985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31916.log 2024-03-21T06:39:39.024 INFO:teuthology.orchestra.run.smithi096.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.31874.log.gz 2024-03-21T06:39:39.024 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34010.log 2024-03-21T06:39:39.024 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.31916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97154.log 2024-03-21T06:39:39.024 INFO:teuthology.orchestra.run.smithi096.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.31916.log.gz 2024-03-21T06:39:39.024 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99264.log 2024-03-21T06:39:39.024 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99075.log 2024-03-21T06:39:39.024 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97154.log: /var/log/ceph/ceph-client.admin.99264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30684.log 2024-03-21T06:39:39.024 INFO:teuthology.orchestra.run.smithi096.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.97154.log.gz 2024-03-21T06:39:39.024 INFO:teuthology.orchestra.run.smithi096.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.95985.log.gz 2024-03-21T06:39:39.024 INFO:teuthology.orchestra.run.smithi096.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99264.log.gz 2024-03-21T06:39:39.024 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30868.log 2024-03-21T06:39:39.024 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.30684.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99075.log.gz 2024-03-21T06:39:39.025 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33842.log 2024-03-21T06:39:39.025 INFO:teuthology.orchestra.run.smithi096.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.30684.log.gz 2024-03-21T06:39:39.025 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.30868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96862.log 2024-03-21T06:39:39.025 INFO:teuthology.orchestra.run.smithi096.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.30868.log.gz 2024-03-21T06:39:39.025 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97003.log 2024-03-21T06:39:39.025 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98709.log 2024-03-21T06:39:39.025 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96862.log: /var/log/ceph/ceph-client.admin.97003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95948.log 2024-03-21T06:39:39.025 INFO:teuthology.orchestra.run.smithi096.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.33842.log.gz 2024-03-21T06:39:39.025 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99703.log 2024-03-21T06:39:39.025 INFO:teuthology.orchestra.run.smithi096.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96862.log.gz 2024-03-21T06:39:39.025 INFO:teuthology.orchestra.run.smithi096.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98709.log.gz 2024-03-21T06:39:39.025 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.95948.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97003.log.gz 2024-03-21T06:39:39.025 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96177.log 2024-03-21T06:39:39.025 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32336.log 2024-03-21T06:39:39.025 INFO:teuthology.orchestra.run.smithi096.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.99703.log.gz 2024-03-21T06:39:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99871.log 2024-03-21T06:39:39.026 INFO:teuthology.orchestra.run.smithi096.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.95948.log.gz 2024-03-21T06:39:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99159.log 2024-03-21T06:39:39.026 INFO:teuthology.orchestra.run.smithi096.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.32336.log.gz 2024-03-21T06:39:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30605.log 2024-03-21T06:39:39.026 INFO:teuthology.orchestra.run.smithi096.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.99871.log.gz 2024-03-21T06:39:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99619.log 2024-03-21T06:39:39.026 INFO:teuthology.orchestra.run.smithi096.stderr: 87.8%/var/log/ceph/ceph-client.admin.30605.log: -- replaced with /var/log/ceph/ceph-client.admin.99159.log.gz 2024-03-21T06:39:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96158.log 2024-03-21T06:39:39.026 INFO:teuthology.orchestra.run.smithi096.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.30605.log.gz 2024-03-21T06:39:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99493.log 2024-03-21T06:39:39.026 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96158.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.99619.log.gz 2024-03-21T06:39:39.026 INFO:teuthology.orchestra.run.smithi096.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.96177.log.gz 2024-03-21T06:39:39.027 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31832.log 2024-03-21T06:39:39.027 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99787.log 2024-03-21T06:39:39.027 INFO:teuthology.orchestra.run.smithi096.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.99493.log.gz 2024-03-21T06:39:39.027 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.31832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98604.log 2024-03-21T06:39:39.027 INFO:teuthology.orchestra.run.smithi096.stderr: 86.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.31832.log.gz 2024-03-21T06:39:39.027 INFO:teuthology.orchestra.run.smithi096.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.97670.log 2024-03-21T06:39:39.027 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99787.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.96158.log.gz 2024-03-21T06:39:39.027 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99331.log 2024-03-21T06:39:39.027 INFO:teuthology.orchestra.run.smithi096.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.99787.log.gz 2024-03-21T06:39:39.027 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97670.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98604.log.gz 2024-03-21T06:39:39.027 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98578.log 2024-03-21T06:39:39.027 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32294.log 2024-03-21T06:39:39.027 INFO:teuthology.orchestra.run.smithi096.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.99331.log.gz 2024-03-21T06:39:39.027 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96120.log 2024-03-21T06:39:39.027 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96711.log 2024-03-21T06:39:39.028 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32294.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.98578.log.gz 2024-03-21T06:39:39.028 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97190.log 2024-03-21T06:39:39.028 INFO:teuthology.orchestra.run.smithi096.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.32294.log.gz 2024-03-21T06:39:39.028 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96711.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96120.log.gz -5 2024-03-21T06:39:39.028 INFO:teuthology.orchestra.run.smithi096.stderr: --verbose -- /var/log/ceph/ceph-client.admin.99307.log 2024-03-21T06:39:39.028 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97264.log 2024-03-21T06:39:39.028 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97190.log: /var/log/ceph/ceph-client.admin.99307.log: gzip 86.2% -- replaced with /var/log/ceph/ceph-client.admin.97190.log.gz 2024-03-21T06:39:39.028 INFO:teuthology.orchestra.run.smithi096.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.32016.log 2024-03-21T06:39:39.028 INFO:teuthology.orchestra.run.smithi096.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.99307.log.gz 2024-03-21T06:39:39.028 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98952.log 2024-03-21T06:39:39.028 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32016.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.97264.log.gz 92.9% 2024-03-21T06:39:39.028 INFO:teuthology.orchestra.run.smithi096.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97670.log.gz 2024-03-21T06:39:39.028 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98171.log 2024-03-21T06:39:39.028 INFO:teuthology.orchestra.run.smithi096.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.32016.log.gz 2024-03-21T06:39:39.029 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97262.log 2024-03-21T06:39:39.029 INFO:teuthology.orchestra.run.smithi096.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.98952.log.gz 2024-03-21T06:39:39.029 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96900.log 2024-03-21T06:39:39.029 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98339.log 2024-03-21T06:39:39.029 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97262.log: /var/log/ceph/ceph-client.admin.96900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99180.log 2024-03-21T06:39:39.029 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97315.log 2024-03-21T06:39:39.029 INFO:teuthology.orchestra.run.smithi096.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97262.log.gz 2024-03-21T06:39:39.029 INFO:teuthology.orchestra.run.smithi096.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.96900.log.gz 2024-03-21T06:39:39.029 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99180.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98339.log.gz 2024-03-21T06:39:39.029 INFO:teuthology.orchestra.run.smithi096.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.99138.log 2024-03-21T06:39:39.029 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96632.log 2024-03-21T06:39:39.029 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99138.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99180.log.gz 2024-03-21T06:39:39.029 INFO:teuthology.orchestra.run.smithi096.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97315.log.gz 2024-03-21T06:39:39.029 INFO:teuthology.orchestra.run.smithi096.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99138.log.gz 2024-03-21T06:39:39.029 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97411.log 2024-03-21T06:39:39.030 INFO:teuthology.orchestra.run.smithi096.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.98171.log.gz 2024-03-21T06:39:39.030 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97004.log 2024-03-21T06:39:39.030 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96632.log: /var/log/ceph/ceph-client.admin.97411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99096.log 2024-03-21T06:39:39.030 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99356.log 2024-03-21T06:39:39.030 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99997.log 2024-03-21T06:39:39.030 INFO:teuthology.orchestra.run.smithi096.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.96632.log.gz 2024-03-21T06:39:39.030 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99356.log: 87.8% 89.9% -- replaced with /var/log/ceph/ceph-client.admin.97004.log.gz -- replaced with /var/log/ceph/ceph-client.admin.99096.log.gz 2024-03-21T06:39:39.030 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-03-21T06:39:39.030 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-21T06:39:39.030 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32015.log 2024-03-21T06:39:39.030 INFO:teuthology.orchestra.run.smithi096.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.99356.log.gz 2024-03-21T06:39:39.030 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96101.log 2024-03-21T06:39:39.030 INFO:teuthology.orchestra.run.smithi096.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.99997.log.gz 2024-03-21T06:39:39.030 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5/var/log/ceph/ceph-client.admin.32015.log: --verbose -- /var/log/ceph/ceph-client.admin.97085.log 2024-03-21T06:39:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96101.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.32015.log.gz 2024-03-21T06:39:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95657.log 2024-03-21T06:39:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96101.log.gz 2024-03-21T06:39:39.031 INFO:teuthology.orchestra.run.smithi096.stderr: --verbose -- /var/log/ceph/ceph-client.admin.99829.log 2024-03-21T06:39:39.031 INFO:teuthology.orchestra.run.smithi096.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.97411.log.gz/var/log/ceph/ceph-client.admin.97085.log: /var/log/ceph/ceph-client.admin.95657.log: 2024-03-21T06:39:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97394.log 2024-03-21T06:39:39.031 INFO:teuthology.orchestra.run.smithi096.stderr: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-21T06:39:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97959.log 2024-03-21T06:39:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97394.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.99829.log.gz 2024-03-21T06:39:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98371.log 2024-03-21T06:39:39.031 INFO:teuthology.orchestra.run.smithi096.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97085.log.gz 2024-03-21T06:39:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97959.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.97394.log.gz 2024-03-21T06:39:39.031 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98817.log 2024-03-21T06:39:39.031 INFO:teuthology.orchestra.run.smithi096.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.96711.log.gz 2024-03-21T06:39:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97347.log 2024-03-21T06:39:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97153.log 2024-03-21T06:39:39.032 INFO:teuthology.orchestra.run.smithi096.stderr: 86.5%/var/log/ceph/ceph-client.admin.97347.log: -- replaced with /var/log/ceph/ceph-client.admin.98817.log.gz 2024-03-21T06:39:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33069.log 2024-03-21T06:39:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32598.log 2024-03-21T06:39:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33069.log: /var/log/ceph/ceph-client.admin.97153.log: 91.0% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97153.log.gz 2024-03-21T06:39:39.032 INFO:teuthology.orchestra.run.smithi096.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97347.log.gz 2024-03-21T06:39:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98730.log 2024-03-21T06:39:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31790.log 2024-03-21T06:39:39.032 INFO:teuthology.orchestra.run.smithi096.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.97959.log.gz 2024-03-21T06:39:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98730.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98730.log.gz 2024-03-21T06:39:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99535.log 2024-03-21T06:39:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-21T06:39:39.032 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99535.log: /var/log/ceph/ceph-client.admin.31790.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.31790.log.gz 2024-03-21T06:39:39.032 INFO:teuthology.orchestra.run.smithi096.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.99535.log.gz 2024-03-21T06:39:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97189.log 2024-03-21T06:39:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100039.log 2024-03-21T06:39:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.97189.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97189.log.gz 2024-03-21T06:39:39.033 INFO:teuthology.orchestra.run.smithi096.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.32598.log.gz 2024-03-21T06:39:39.033 INFO:teuthology.orchestra.run.smithi096.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.98371.log.gz 2024-03-21T06:39:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97002.log 2024-03-21T06:39:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.100039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99201.log 2024-03-21T06:39:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97002.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.100039.log.gz 2024-03-21T06:39:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99286.log 2024-03-21T06:39:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99201.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97002.log.gz 2024-03-21T06:39:39.033 INFO:teuthology.orchestra.run.smithi096.stderr: 95.3% -- replaced with /var/log/ceph/ceph-client.admin.95657.log.gz 2024-03-21T06:39:39.033 INFO:teuthology.orchestra.run.smithi096.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99201.log.gz 2024-03-21T06:39:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99243.log 2024-03-21T06:39:39.033 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97263.log 2024-03-21T06:39:39.034 INFO:teuthology.orchestra.run.smithi096.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99286.log.gz 2024-03-21T06:39:39.034 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96899.log 2024-03-21T06:39:39.034 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97263.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99243.log.gz 2024-03-21T06:39:39.034 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32184.log 2024-03-21T06:39:39.034 INFO:teuthology.orchestra.run.smithi096.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.97263.log.gz 2024-03-21T06:39:39.034 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33991.log 2024-03-21T06:39:39.034 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96899.log: /var/log/ceph/ceph-client.admin.32184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99955.log 2024-03-21T06:39:39.034 INFO:teuthology.orchestra.run.smithi096.stderr: 90.7%/var/log/ceph/ceph-client.admin.33991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98517.log 2024-03-21T06:39:39.034 INFO:teuthology.orchestra.run.smithi096.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.96899.log.gz 2024-03-21T06:39:39.034 INFO:teuthology.orchestra.run.smithi096.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32184.log.gz 2024-03-21T06:39:39.034 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99955.log: gzip -5 --verbose -- 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33991.log.gz 2024-03-21T06:39:39.034 INFO:teuthology.orchestra.run.smithi096.stderr: /var/log/ceph/ceph-client.admin.96898.log 2024-03-21T06:39:39.035 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98625.log 2024-03-21T06:39:39.035 INFO:teuthology.orchestra.run.smithi096.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.98517.log.gz 85.5% 2024-03-21T06:39:39.035 INFO:teuthology.orchestra.run.smithi096.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.33069.log.gz 2024-03-21T06:39:39.035 INFO:teuthology.orchestra.run.smithi096.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99955.log.gz 2024-03-21T06:39:39.035 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96838.log 2024-03-21T06:39:39.035 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98543.log 2024-03-21T06:39:39.035 INFO:teuthology.orchestra.run.smithi096.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98625.log.gz 2024-03-21T06:39:39.035 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96838.log: gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96898.log.gz 2024-03-21T06:39:39.035 INFO:teuthology.orchestra.run.smithi096.stderr: --verbose -- /var/log/ceph/ceph-client.admin.31748.log 2024-03-21T06:39:39.035 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98543.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.98543.log.gz 2024-03-21T06:39:39.035 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98816.log 2024-03-21T06:39:39.035 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99913.log 2024-03-21T06:39:39.035 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.31748.log: /var/log/ceph/ceph-client.admin.98816.log: gzip 85.9% -- replaced with /var/log/ceph/ceph-client.admin.31748.log.gz 2024-03-21T06:39:39.035 INFO:teuthology.orchestra.run.smithi096.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.96951.log 2024-03-21T06:39:39.035 INFO:teuthology.orchestra.run.smithi096.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.96838.log.gz 2024-03-21T06:39:39.036 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99913.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98816.log.gz 2024-03-21T06:39:39.036 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98793.log 2024-03-21T06:39:39.036 INFO:teuthology.orchestra.run.smithi096.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.99913.log.gz 2024-03-21T06:39:39.036 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99222.log 2024-03-21T06:39:39.036 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98772.log 2024-03-21T06:39:39.036 INFO:teuthology.orchestra.run.smithi096.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98793.log.gz 2024-03-21T06:39:39.036 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33972.log 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96951.log.gz 2024-03-21T06:39:39.036 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-03-21T06:39:39.036 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98772.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99222.log.gz 2024-03-21T06:39:39.036 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96861.log 2024-03-21T06:39:39.036 INFO:teuthology.orchestra.run.smithi096.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98772.log.gz 2024-03-21T06:39:39.036 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33876.log 2024-03-21T06:39:39.036 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33972.log: /var/log/ceph/ceph-client.admin.96861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97001.log 2024-03-21T06:39:39.036 INFO:teuthology.orchestra.run.smithi096.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33972.log.gz 2024-03-21T06:39:39.037 INFO:teuthology.orchestra.run.smithi096.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.96861.log.gz 2024-03-21T06:39:39.037 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30840.log 2024-03-21T06:39:39.037 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96863.log 2024-03-21T06:39:39.037 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.30840.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.97001.log.gz 2024-03-21T06:39:39.037 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98907.log 2024-03-21T06:39:39.037 INFO:teuthology.orchestra.run.smithi096.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.33876.log.gz 2024-03-21T06:39:39.037 INFO:teuthology.orchestra.run.smithi096.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.30840.log.gz 2024-03-21T06:39:39.037 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32185.log 2024-03-21T06:39:39.037 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97852.log 2024-03-21T06:39:39.037 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98907.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.96863.log.gz 2024-03-21T06:39:39.037 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32185.log: gzip -5 --verbose 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98907.log.gz 2024-03-21T06:39:39.037 INFO:teuthology.orchestra.run.smithi096.stderr: -- /var/log/ceph/ceph-client.admin.32018.log 2024-03-21T06:39:39.037 INFO:teuthology.orchestra.run.smithi096.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.32185.log.gz 2024-03-21T06:39:39.037 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98908.log 2024-03-21T06:39:39.037 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98751.log 2024-03-21T06:39:39.038 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98908.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.32018.log.gz 2024-03-21T06:39:39.038 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33934.log 2024-03-21T06:39:39.038 INFO:teuthology.orchestra.run.smithi096.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.97852.log.gz 2024-03-21T06:39:39.038 INFO:teuthology.orchestra.run.smithi096.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.98908.log.gz 2024-03-21T06:39:39.038 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99048.log 2024-03-21T06:39:39.038 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33934.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98751.log.gz 2024-03-21T06:39:39.038 INFO:teuthology.orchestra.run.smithi096.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.97623.log 2024-03-21T06:39:39.038 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99745.log 2024-03-21T06:39:39.038 INFO:teuthology.orchestra.run.smithi096.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.99048.log.gz 2024-03-21T06:39:39.038 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97087.log 2024-03-21T06:39:39.038 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96244.log 2024-03-21T06:39:39.038 INFO:teuthology.orchestra.run.smithi096.stderr: 86.6%/var/log/ceph/ceph-client.admin.97087.log: -- replaced with /var/log/ceph/ceph-client.admin.99745.log.gz 2024-03-21T06:39:39.038 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-21T06:39:39.038 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96139.log 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97087.log.gz 2024-03-21T06:39:39.039 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-03-21T06:39:39.039 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100079.log 2024-03-21T06:39:39.039 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96139.log: /var/log/ceph/ceph-osd.0.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.96244.log.gz 2024-03-21T06:39:39.039 INFO:teuthology.orchestra.run.smithi096.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.96139.log.gz 2024-03-21T06:39:39.039 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98882.log 2024-03-21T06:39:39.039 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.100079.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.100079.log.gz 2024-03-21T06:39:39.039 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98976.log 2024-03-21T06:39:39.039 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96950.log 2024-03-21T06:39:39.039 INFO:teuthology.orchestra.run.smithi096.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.98882.log.gz 2024-03-21T06:39:39.039 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95729.log 2024-03-21T06:39:39.039 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96950.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98976.log.gz 2024-03-21T06:39:39.039 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33895.log 2024-03-21T06:39:39.039 INFO:teuthology.orchestra.run.smithi096.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.96950.log.gz 2024-03-21T06:39:39.039 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.95729.log: 92.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32183.log 2024-03-21T06:39:39.039 INFO:teuthology.orchestra.run.smithi096.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97623.log.gz 2024-03-21T06:39:39.040 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33895.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95729.log.gz 2024-03-21T06:39:39.040 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95767.log 2024-03-21T06:39:39.040 INFO:teuthology.orchestra.run.smithi096.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.33895.log.gz 2024-03-21T06:39:39.040 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32183.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-21T06:39:39.040 INFO:teuthology.orchestra.run.smithi096.stderr: 86.1%/var/log/ceph/ceph-client.admin.95767.log: -- replaced with /var/log/ceph/ceph-client.admin.32183.log.gz 2024-03-21T06:39:39.040 INFO:teuthology.orchestra.run.smithi096.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.33934.log.gz 2024-03-21T06:39:39.040 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32017.log 2024-03-21T06:39:39.040 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99380.log 2024-03-21T06:39:39.040 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32378.log 2024-03-21T06:39:39.040 INFO:teuthology.orchestra.run.smithi096.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.32017.log.gz 2024-03-21T06:39:39.040 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99024.log 92.0% -- replaced with /var/log/ceph/ceph-client.admin.95767.log.gz 2024-03-21T06:39:39.040 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-03-21T06:39:39.040 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32378.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.99380.log.gz 2024-03-21T06:39:39.040 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97000.log 2024-03-21T06:39:39.041 INFO:teuthology.orchestra.run.smithi096.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.32378.log.gz 2024-03-21T06:39:39.041 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98646.log 2024-03-21T06:39:39.041 INFO:teuthology.orchestra.run.smithi096.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.99024.log.gz 2024-03-21T06:39:39.041 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28296.log 2024-03-21T06:39:39.041 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97000.log: /var/log/ceph/ceph-client.admin.98646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98998.log 2024-03-21T06:39:39.041 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph.tmp-client.admin.28296.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98646.log.gz 2024-03-21T06:39:39.041 INFO:teuthology.orchestra.run.smithi096.stderr: 68.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28296.log.gz 2024-03-21T06:39:39.041 INFO:teuthology.orchestra.run.smithi096.stderr: 87.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-21T06:39:39.041 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98861.log 2024-03-21T06:39:39.041 INFO:teuthology.orchestra.run.smithi096.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97000.log.gz 2024-03-21T06:39:39.041 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99117.log 2024-03-21T06:39:39.041 INFO:teuthology.orchestra.run.smithi096.stderr: 86.1%/var/log/ceph/ceph-client.admin.98861.log: -- replaced with /var/log/ceph/ceph-client.admin.98998.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99577.log 2024-03-21T06:39:39.041 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-03-21T06:39:39.041 INFO:teuthology.orchestra.run.smithi096.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98861.log.gz 2024-03-21T06:39:39.041 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98667.log 2024-03-21T06:39:39.042 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 87.8% --verbose -- /var/log/ceph/ceph-client.admin.32186.log 2024-03-21T06:39:39.042 INFO:teuthology.orchestra.run.smithi096.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99117.log.gz 2024-03-21T06:39:39.042 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99577.log: /var/log/ceph/ceph-client.admin.98667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31684.log 2024-03-21T06:39:39.042 INFO:teuthology.orchestra.run.smithi096.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.99577.log.gz 2024-03-21T06:39:39.042 INFO:teuthology.orchestra.run.smithi096.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98667.log.gz 2024-03-21T06:39:39.042 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96949.log 2024-03-21T06:39:39.042 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32186.log: /var/log/ceph/ceph-client.admin.31684.log: gzip -5 --verbose 86.4% -- replaced with /var/log/ceph/ceph-client.admin.32186.log.gz 2024-03-21T06:39:39.042 INFO:teuthology.orchestra.run.smithi096.stderr: -- /var/log/ceph/ceph-client.admin.96857.log 2024-03-21T06:39:39.042 INFO:teuthology.orchestra.run.smithi096.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.31684.log.gz 2024-03-21T06:39:39.042 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97226.log 2024-03-21T06:39:39.042 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96857.log: gzip -5 66.7% -- replaced with /var/log/ceph/ceph-client.admin.96857.log.gz 2024-03-21T06:39:39.042 INFO:teuthology.orchestra.run.smithi096.stderr: --verbose -- /var/log/ceph/ceph-client.admin.97227.log 2024-03-21T06:39:39.042 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97226.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96949.log.gz 2024-03-21T06:39:39.043 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97086.log 2024-03-21T06:39:39.043 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97227.log: gzip -5 --verbose -- 85.9% -- replaced with /var/log/ceph/ceph-client.admin.97227.log.gz 2024-03-21T06:39:39.043 INFO:teuthology.orchestra.run.smithi096.stderr: /var/log/ceph/ceph-client.admin.99661.log 2024-03-21T06:39:39.043 INFO:teuthology.orchestra.run.smithi096.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97226.log.gz 2024-03-21T06:39:39.043 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.97086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96295.log 2024-03-21T06:39:39.043 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.99661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98688.log 2024-03-21T06:39:39.043 INFO:teuthology.orchestra.run.smithi096.stderr: 90.4% 86.7% -- replaced with /var/log/ceph/ceph-client.admin.99661.log.gz 2024-03-21T06:39:39.043 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.96295.log: -- replaced with /var/log/ceph/ceph-client.admin.97086.log.gz 2024-03-21T06:39:39.043 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.98688.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98688.log.gz 2024-03-21T06:39:39.043 INFO:teuthology.orchestra.run.smithi096.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.96295.log.gz 2024-03-21T06:39:39.166 INFO:teuthology.orchestra.run.smithi184.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-21T06:39:39.407 INFO:teuthology.orchestra.run.smithi096.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.34010.log.gz 2024-03-21T06:39:40.452 INFO:teuthology.orchestra.run.smithi096.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-21T06:39:43.664 INFO:teuthology.orchestra.run.smithi096.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-21T06:39:44.028 INFO:teuthology.orchestra.run.smithi184.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-21T06:39:44.054 INFO:teuthology.orchestra.run.smithi184.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-21T06:39:44.056 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-03-21T06:39:44.056 INFO:teuthology.orchestra.run.smithi184.stderr:real 0m5.121s 2024-03-21T06:39:44.056 INFO:teuthology.orchestra.run.smithi184.stderr:user 0m9.941s 2024-03-21T06:39:44.056 INFO:teuthology.orchestra.run.smithi184.stderr:sys 0m0.466s 2024-03-21T06:39:44.463 INFO:teuthology.orchestra.run.smithi096.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-21T06:39:44.465 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-03-21T06:39:44.465 INFO:teuthology.orchestra.run.smithi096.stderr:real 0m5.534s 2024-03-21T06:39:44.465 INFO:teuthology.orchestra.run.smithi096.stderr:user 0m11.798s 2024-03-21T06:39:44.465 INFO:teuthology.orchestra.run.smithi096.stderr:sys 0m0.579s 2024-03-21T06:39:44.465 INFO:tasks.ceph:Archiving logs... 2024-03-21T06:39:44.466 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi/7614674/remote/smithi096/log 2024-03-21T06:39:44.466 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T06:39:46.546 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi/7614674/remote/smithi184/log 2024-03-21T06:39:46.547 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T06:39:48.129 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-21T06:39:48.138 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-21T06:39:48.138 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T06:39:48.141 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T06:39:48.232 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-21T06:39:48.241 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-21T06:39:48.241 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T06:39:48.244 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T06:39:48.257 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-03-21T06:39:48.260 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-03-21T06:39:48.261 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T06:39:48.261 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-03-21T06:39:48.261 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 107 -160us[ -137us] +/- 74ms 2024-03-21T06:39:48.261 INFO:teuthology.orchestra.run.smithi096.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 -1198us[-1193us] +/- 74ms 2024-03-21T06:39:48.261 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 +1434us[+1439us] +/- 75ms 2024-03-21T06:39:48.262 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T06:39:48.264 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T06:39:48.264 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-03-21T06:39:48.264 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 44 -138us[ -138us] +/- 75ms 2024-03-21T06:39:48.264 INFO:teuthology.orchestra.run.smithi184.stdout:^* hv02.front.sepia.ceph.com 3 6 377 46 -1146us[-1152us] +/- 74ms 2024-03-21T06:39:48.264 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 +1480us[+1480us] +/- 75ms 2024-03-21T06:39:48.264 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T06:39:48.265 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-21T06:39:48.274 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-21T06:39:48.274 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-21T06:39:48.322 DEBUG:teuthology.orchestra.run.smithi096:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T06:39:48.393 DEBUG:teuthology.orchestra.run.smithi184:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T06:39:48.463 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T06:39:48.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T06:39:48.495 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T06:39:48.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T06:39:48.527 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-21T06:39:48.536 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-21T06:39:48.545 INFO:teuthology.task.internal:Duration was 1986.384081 seconds 2024-03-21T06:39:48.546 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-21T06:39:48.554 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-21T06:39:48.555 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T06:39:48.558 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T06:39:48.600 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T06:39:48.613 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T06:39:48.905 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-21T06:39:48.905 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-03-21T06:39:48.905 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T06:39:48.927 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2024-03-21T06:39:48.927 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T06:39:49.066 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-21T06:39:49.067 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T06:39:49.069 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T06:39:49.101 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-21T06:39:49.101 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T06:39:49.140 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T06:39:49.239 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-21T06:39:49.256 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-21T06:39:49.256 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T06:39:49.285 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T06:39:49.312 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-21T06:39:49.327 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T06:39:49.330 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T06:39:49.353 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-03-21T06:39:49.378 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2024-03-21T06:39:49.415 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T06:39:49.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T06:39:49.429 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T06:39:49.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T06:39:49.469 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-21T06:39:49.479 INFO:teuthology.task.internal:Transferring archived files... 2024-03-21T06:39:49.480 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi/7614674/remote/smithi096 2024-03-21T06:39:49.481 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T06:39:49.516 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi/7614674/remote/smithi184 2024-03-21T06:39:49.517 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T06:39:49.552 INFO:teuthology.task.internal:Removing archive directory... 2024-03-21T06:39:49.553 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T06:39:49.555 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T06:39:49.601 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-21T06:39:49.613 INFO:teuthology.task.internal:Not uploading archives. 2024-03-21T06:39:49.613 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-21T06:39:49.622 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-21T06:39:49.623 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T06:39:49.625 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T06:39:49.637 INFO:teuthology.orchestra.run.smithi096.stdout: 265917 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 06:39 /home/ubuntu/cephtest 2024-03-21T06:39:49.656 INFO:teuthology.orchestra.run.smithi184.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 06:39 /home/ubuntu/cephtest 2024-03-21T06:39:49.657 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-21T06:39:49.667 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-21T06:39:49.688 INFO:teuthology.nuke:Checking targets against current locks 2024-03-21T06:39:49.718 DEBUG:teuthology.nuke:shortname: smithi096 2024-03-21T06:39:49.719 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T06:39:49.739 DEBUG:teuthology.nuke:shortname: smithi184 2024-03-21T06:39:49.740 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T06:39:49.747 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi/7614674', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 05:52:48.676040', 'locked_by': 'scheduled_yingxin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIUHKbEkR+HUbnaQRi1xHhA4XWyrvOa7J+DYI/mzrXugPbvImI7V4WIeSTLo+3Bm10E3FBR8P1PMH2Py0TAZ+vs='} 2024-03-21T06:39:49.760 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yingxin-2024-03-21_05:50:24-crimson-rados-ci-yingxin-fix-racing-get-or-create-pg-mapping-2-distro-default-smithi/7614674', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 05:52:48.677848', 'locked_by': 'scheduled_yingxin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBPpXxYSfuuxjw/v/WXhHe553KpRC/nY5aQTyIFm3OYUyj9OKGREjr66EM1brOjwyTGoe4KJHdb0kzRgfN1wrak='} 2024-03-21T06:39:49.788 INFO:teuthology.orchestra.console.smithi096:Power off 2024-03-21T06:39:49.788 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T06:39:49.812 DEBUG:teuthology.orchestra.console.smithi096:power off output: Chassis Power Control: Down/Off 2024-03-21T06:39:49.813 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T06:39:49.830 INFO:teuthology.orchestra.console.smithi184:Power off 2024-03-21T06:39:49.830 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T06:39:49.844 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on 2024-03-21T06:39:49.857 DEBUG:teuthology.orchestra.console.smithi184:power off output: Chassis Power Control: Down/Off 2024-03-21T06:39:49.857 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T06:39:49.881 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2024-03-21T06:39:53.845 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T06:39:53.882 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T06:39:53.956 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on 2024-03-21T06:39:53.992 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2024-03-21T06:39:57.957 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T06:39:57.992 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T06:39:58.071 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on 2024-03-21T06:39:58.103 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2024-03-21T06:40:02.075 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T06:40:02.104 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T06:40:02.187 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is off 2024-03-21T06:40:02.216 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is off 2024-03-21T06:40:02.289 INFO:teuthology.orchestra.console.smithi096:Power off completed 2024-03-21T06:40:02.317 INFO:teuthology.orchestra.console.smithi184:Power off completed 2024-03-21T06:40:02.422 INFO:teuthology.run:Summary data: description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} duration: 1986.384081363678 failure_reason: '"2024-03-21T06:39:05.491020+0000 mon.a (mon.0) 1342 : cluster [ERR] Health check failed: 23 scrub errors (OSD_SCRUB_ERRORS)" in cluster log' flavor: crimson owner: scheduled_yingxin@teuthology success: false 2024-03-21T06:40:02.422 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T06:40:02.509 INFO:teuthology.run:FAIL