2024-09-22T03:36:01.338 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-22T03:36:01.354 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-22T03:36:01.437 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi/7914369 branch: main description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore tasks/rados_api_tests} email: ceph-qa@ceph.com first_in_suite: false job_id: '7914369' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: debug ms: 1 global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd pool default crimson: true osd: crimson alien thread cpu cores: 6-7 crimson osd obc lru size: 10 debug ms: 1 debug osd: 20 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore ompap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd blocked scrub grace period: 3600 osd class default list: '*' osd class load list: '*' osd objectstore: seastore osd shutdown pgref assert: true osd.0: crimson seastar cpu cores: 0-2 osd.1: crimson seastar cpu cores: 3-5 osd.2: crimson seastar cpu cores: 0-2 osd.3: crimson seastar cpu cores: 3-5 flavor: crimson fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 62da54b4b83a96ca9976b7e1d0c8272a564a2208 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: crimson sha1: 62da54b4b83a96ca9976b7e1d0c8272a564a2208 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: 62da54b4b83a96ca9976b7e1d0c8272a564a2208 owner: scheduled_teuthology@teuthology priority: 101 repo: https://git.ceph.com/ceph.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b seed: 2950 sha1: 62da54b4b83a96ca9976b7e1d0c8272a564a2208 sleep_before_teardown: 0 subset: 0/1 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_62da54b4b83a96ca9976b7e1d0c8272a564a2208/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 62da54b4b83a96ca9976b7e1d0c8272a564a2208 targets: smithi003.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNHcqJqBGP0EcDBMWF44nqaSlIl/Jzt0sq7WqIPSoJ+A1Yf9sko32i9UA7DoeM5QJQc60THmcnGvOz5a3SYGIH8= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDXtdxge8lX322TMSHVbyu23N4c3Zp9vEfKmeXoriZlRHiOP5d0CH3Vtq5QMUO2rwD15oBhvQ0XU8upeNBnxXSM= tasks: - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 flavor: crimson - workunit: clients: client.0: - rados/test.sh --crimson - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-21_20:56:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-22T03:36:01.437 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_62da54b4b83a96ca9976b7e1d0c8272a564a2208/qa; will attempt to use it 2024-09-22T03:36:01.438 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_62da54b4b83a96ca9976b7e1d0c8272a564a2208/qa/tasks 2024-09-22T03:36:01.438 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-22T03:36:01.439 INFO:teuthology.task.internal:Checking packages... 2024-09-22T03:36:01.460 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash '62da54b4b83a96ca9976b7e1d0c8272a564a2208' 2024-09-22T03:36:01.460 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-22T03:36:01.460 INFO:teuthology.packaging:ref: None 2024-09-22T03:36:01.460 INFO:teuthology.packaging:tag: None 2024-09-22T03:36:01.461 INFO:teuthology.packaging:branch: main 2024-09-22T03:36:01.461 INFO:teuthology.packaging:sha1: 62da54b4b83a96ca9976b7e1d0c8272a564a2208 2024-09-22T03:36:01.461 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&ref=main 2024-09-22T03:36:01.658 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5092.g62da54b4 2024-09-22T03:36:01.660 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-22T03:36:01.678 INFO:teuthology.task.internal:no buildpackages task found 2024-09-22T03:36:01.678 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-22T03:36:01.719 INFO:teuthology.task.internal:Saving configuration 2024-09-22T03:36:01.730 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-22T03:36:01.744 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-22T03:36:01.768 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi/7914369', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-22 03:31:11.500561', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNHcqJqBGP0EcDBMWF44nqaSlIl/Jzt0sq7WqIPSoJ+A1Yf9sko32i9UA7DoeM5QJQc60THmcnGvOz5a3SYGIH8='} 2024-09-22T03:36:01.788 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi/7914369', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-22 03:31:11.501851', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDXtdxge8lX322TMSHVbyu23N4c3Zp9vEfKmeXoriZlRHiOP5d0CH3Vtq5QMUO2rwD15oBhvQ0XU8upeNBnxXSM='} 2024-09-22T03:36:01.788 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-22T03:36:01.794 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2024-09-22T03:36:01.794 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-22T03:36:01.794 INFO:teuthology.run_tasks:Running task console_log... 2024-09-22T03:36:01.961 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f813de3c160>, signals=[15]) 2024-09-22T03:36:01.961 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-22T03:36:01.995 INFO:teuthology.task.internal:Opening connections... 2024-09-22T03:36:01.995 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2024-09-22T03:36:01.996 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:36:02.073 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-09-22T03:36:02.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:36:02.145 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-22T03:36:02.152 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2024-09-22T03:36:02.170 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2024-09-22T03:36:02.171 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2024-09-22T03:36:02.228 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="CentOS Stream" 2024-09-22T03:36:02.228 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="9" 2024-09-22T03:36:02.228 INFO:teuthology.orchestra.run.smithi003.stdout:ID="centos" 2024-09-22T03:36:02.228 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="rhel fedora" 2024-09-22T03:36:02.229 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="9" 2024-09-22T03:36:02.229 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el9" 2024-09-22T03:36:02.229 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-22T03:36:02.229 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31" 2024-09-22T03:36:02.229 INFO:teuthology.orchestra.run.smithi003.stdout:LOGO="fedora-logo-icon" 2024-09-22T03:36:02.229 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-22T03:36:02.229 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://centos.org/" 2024-09-22T03:36:02.229 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-22T03:36:02.229 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-22T03:36:02.229 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-22T03:36:02.230 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2024-09-22T03:36:02.248 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-09-22T03:36:02.265 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-09-22T03:36:02.265 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-09-22T03:36:02.323 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-09-22T03:36:02.323 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-09-22T03:36:02.323 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-09-22T03:36:02.323 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-09-22T03:36:02.323 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-09-22T03:36:02.323 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-09-22T03:36:02.323 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-22T03:36:02.323 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-09-22T03:36:02.323 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-09-22T03:36:02.323 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-22T03:36:02.323 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-09-22T03:36:02.323 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-22T03:36:02.324 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-22T03:36:02.324 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-22T03:36:02.324 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-09-22T03:36:02.348 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-22T03:36:02.356 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-22T03:36:02.361 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-22T03:36:02.361 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2024-09-22T03:36:02.364 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-09-22T03:36:02.379 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-22T03:36:02.385 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-22T03:36:02.385 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2024-09-22T03:36:02.421 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-09-22T03:36:02.444 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-22T03:36:02.496 INFO:teuthology.run_tasks:Running task kernel... 2024-09-22T03:36:02.510 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-22T03:36:02.510 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-22T03:36:02.511 DEBUG:teuthology.orchestra.run.smithi003:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-22T03:36:02.511 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-22T03:36:02.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:36:02.528 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-09-22T03:36:02.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:36:02.529 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-09-22T03:36:02.584 INFO:teuthology.orchestra.run.smithi123.stdout:5.14.0-503.el9.x86_64 2024-09-22T03:36:02.585 INFO:teuthology.task.kernel:Running kernel on smithi123: 5.14.0-503.el9.x86_64 2024-09-22T03:36:02.585 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum install -y kernel 2024-09-22T03:36:02.586 INFO:teuthology.orchestra.run.smithi003.stdout:5.14.0-503.el9.x86_64 2024-09-22T03:36:02.587 INFO:teuthology.task.kernel:Running kernel on smithi003: 5.14.0-503.el9.x86_64 2024-09-22T03:36:02.587 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum install -y kernel 2024-09-22T03:36:04.782 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.3 MB 00:00 2024-09-22T03:36:06.190 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.3 MB 00:01 2024-09-22T03:36:08.108 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 8.9 MB/s | 20 MB 00:02 2024-09-22T03:36:08.165 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-22T03:36:12.500 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-22T03:36:12.508 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-22T03:36:13.976 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 75 kB/s | 19 kB 00:00 2024-09-22T03:36:14.292 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 19 kB 00:00 2024-09-22T03:36:15.963 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 12 MB/s | 23 MB 00:01 2024-09-22T03:36:16.105 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 13 MB/s | 23 MB 00:01 2024-09-22T03:36:21.335 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-22T03:36:21.727 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-22T03:36:23.224 INFO:teuthology.orchestra.run.smithi123.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-22T03:36:23.225 INFO:teuthology.orchestra.run.smithi123.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-22T03:36:23.301 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:36:23.306 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:36:23.306 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-09-22T03:36:23.307 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:36:23.307 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-22T03:36:23.307 INFO:teuthology.orchestra.run.smithi123.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-22T03:36:23.307 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-22T03:36:23.307 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-22T03:36:23.307 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-22T03:36:23.307 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-22T03:36:23.307 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:36:23.307 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-22T03:36:23.307 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:36:23.307 INFO:teuthology.orchestra.run.smithi123.stdout:Install 4 Packages 2024-09-22T03:36:23.308 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:36:23.308 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 79 M 2024-09-22T03:36:23.308 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 126 M 2024-09-22T03:36:23.308 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-22T03:36:23.670 INFO:teuthology.orchestra.run.smithi123.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 329 kB/s | 36 kB 00:00 2024-09-22T03:36:23.709 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-22T03:36:23.709 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-22T03:36:23.803 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:36:23.810 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:36:23.810 INFO:teuthology.orchestra.run.smithi003.stdout: Package Architecture Version Repository Size 2024-09-22T03:36:23.811 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:36:23.811 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-22T03:36:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-22T03:36:23.811 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-09-22T03:36:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-22T03:36:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-22T03:36:23.811 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-22T03:36:23.812 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:36:23.812 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-22T03:36:23.812 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:36:23.812 INFO:teuthology.orchestra.run.smithi003.stdout:Install 4 Packages 2024-09-22T03:36:23.812 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:36:23.813 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 79 M 2024-09-22T03:36:23.813 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 126 M 2024-09-22T03:36:23.813 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-22T03:36:24.020 INFO:teuthology.orchestra.run.smithi003.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 444 kB/s | 36 kB 00:00 2024-09-22T03:36:24.170 INFO:teuthology.orchestra.run.smithi123.stdout:(2/4): kernel-modules-core-5.14.0-508.el9.x86_6 57 MB/s | 29 MB 00:00 2024-09-22T03:36:24.487 INFO:teuthology.orchestra.run.smithi123.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 37 MB/s | 35 MB 00:00 2024-09-22T03:36:24.637 INFO:teuthology.orchestra.run.smithi123.stdout:(4/4): kernel-core-5.14.0-508.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-09-22T03:36:24.637 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:36:24.638 INFO:teuthology.orchestra.run.smithi123.stdout:Total 59 MB/s | 79 MB 00:01 2024-09-22T03:36:25.313 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-22T03:36:25.673 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-22T03:36:25.673 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-22T03:36:26.627 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-22T03:36:26.627 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-22T03:36:27.787 INFO:teuthology.orchestra.run.smithi003.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 4.1 MB/s | 16 MB 00:03 2024-09-22T03:36:29.055 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-22T03:36:29.697 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-22T03:36:29.779 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:36:31.311 INFO:teuthology.orchestra.run.smithi003.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 3.9 MB/s | 29 MB 00:07 2024-09-22T03:36:32.546 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:36:32.665 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:36:32.787 INFO:teuthology.orchestra.run.smithi003.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 3.9 MB/s | 35 MB 00:08 2024-09-22T03:36:32.787 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:36:32.787 INFO:teuthology.orchestra.run.smithi003.stdout:Total 8.8 MB/s | 79 MB 00:08 2024-09-22T03:36:33.503 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-22T03:36:33.893 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-22T03:36:33.893 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-22T03:36:34.857 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-22T03:36:34.858 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-22T03:36:37.416 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-22T03:36:37.472 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:36:37.507 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:36:38.067 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-22T03:36:38.155 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:36:40.892 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:36:41.016 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:36:42.329 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:36:46.142 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:36:46.184 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:36:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:37:20.728 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:37:20.768 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:37:21.058 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:37:21.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-22T03:37:21.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:37:21.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:37:21.876 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:37:21.876 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:37:21.876 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-22T03:37:21.876 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-22T03:37:21.876 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-22T03:37:21.876 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-22T03:37:21.876 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-22T03:37:21.876 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:37:21.876 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:37:22.233 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q kernel | sort -rV | head -n 1 2024-09-22T03:37:22.319 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-22T03:37:22.319 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-22T03:37:22.320 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-22T03:37:22.320 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-22T03:37:22.320 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-22T03:37:22.320 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-09-22T03:37:22.320 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi123.front.sepia.ceph.com, path=None, version=distro) 2024-09-22T03:37:22.320 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q kernel | sort -rV | head -n 1 2024-09-22T03:37:22.379 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-22T03:37:22.379 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-22T03:37:22.379 DEBUG:teuthology.orchestra.run.smithi123:> sudo rpm -qi grub2-tools 2024-09-22T03:37:22.411 INFO:teuthology.orchestra.run.smithi123.stdout:Name : grub2-tools 2024-09-22T03:37:22.411 INFO:teuthology.orchestra.run.smithi123.stdout:Epoch : 1 2024-09-22T03:37:22.411 INFO:teuthology.orchestra.run.smithi123.stdout:Version : 2.06 2024-09-22T03:37:22.411 INFO:teuthology.orchestra.run.smithi123.stdout:Release : 82.el9 2024-09-22T03:37:22.411 INFO:teuthology.orchestra.run.smithi123.stdout:Architecture: x86_64 2024-09-22T03:37:22.412 INFO:teuthology.orchestra.run.smithi123.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-22T03:37:22.412 INFO:teuthology.orchestra.run.smithi123.stdout:Group : Unspecified 2024-09-22T03:37:22.412 INFO:teuthology.orchestra.run.smithi123.stdout:Size : 8274670 2024-09-22T03:37:22.412 INFO:teuthology.orchestra.run.smithi123.stdout:License : GPLv3+ 2024-09-22T03:37:22.412 INFO:teuthology.orchestra.run.smithi123.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-22T03:37:22.412 INFO:teuthology.orchestra.run.smithi123.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-22T03:37:22.412 INFO:teuthology.orchestra.run.smithi123.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-22T03:37:22.412 INFO:teuthology.orchestra.run.smithi123.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-22T03:37:22.412 INFO:teuthology.orchestra.run.smithi123.stdout:Packager : builder@centos.org 2024-09-22T03:37:22.412 INFO:teuthology.orchestra.run.smithi123.stdout:Vendor : CentOS 2024-09-22T03:37:22.412 INFO:teuthology.orchestra.run.smithi123.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-22T03:37:22.412 INFO:teuthology.orchestra.run.smithi123.stdout:Summary : Support tools for GRUB. 2024-09-22T03:37:22.412 INFO:teuthology.orchestra.run.smithi123.stdout:Description : 2024-09-22T03:37:22.413 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:37:22.413 INFO:teuthology.orchestra.run.smithi123.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-22T03:37:22.413 INFO:teuthology.orchestra.run.smithi123.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-22T03:37:22.413 INFO:teuthology.orchestra.run.smithi123.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-22T03:37:22.413 INFO:teuthology.orchestra.run.smithi123.stdout:hardware devices. 2024-09-22T03:37:22.413 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:37:22.413 INFO:teuthology.orchestra.run.smithi123.stdout:This subpackage provides tools for support of all platforms. 2024-09-22T03:37:22.414 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-22T03:37:22.414 INFO:teuthology.task.kernel:Updating grub on smithi123 to boot 5.14.0-508.el9.x86_64 2024-09-22T03:37:22.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-22T03:37:23.112 INFO:teuthology.orchestra.run.smithi123.stderr:Generating grub configuration file ... 2024-09-22T03:37:24.447 INFO:teuthology.orchestra.run.smithi123.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-22T03:37:24.483 INFO:teuthology.orchestra.run.smithi123.stderr:done 2024-09-22T03:37:24.485 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-09-22T03:37:24.500 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.TeT9okKERt 2024-09-22T03:37:24.501 DEBUG:teuthology.orchestra.run.smithi123:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.TeT9okKERt 2024-09-22T03:37:24.567 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.TeT9okKERt 2024-09-22T03:37:24.712 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.TeT9okKERt is 6KB 2024-09-22T03:37:24.761 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.TeT9okKERt 2024-09-22T03:37:24.776 DEBUG:teuthology.orchestra.run.smithi123:> sudo /bin/ls /boot/loader/entries || true 2024-09-22T03:37:24.838 INFO:teuthology.orchestra.run.smithi123.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-22T03:37:24.838 INFO:teuthology.orchestra.run.smithi123.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-22T03:37:24.838 INFO:teuthology.orchestra.run.smithi123.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-22T03:37:24.839 INFO:teuthology.orchestra.run.smithi123.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-22T03:37:24.840 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-22T03:37:24.990 DEBUG:teuthology.orchestra.run.smithi123:> sudo shutdown -r now 2024-09-22T03:37:30.488 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:37:30.529 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:37:30.901 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:37:30.901 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-22T03:37:30.901 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-22T03:37:30.901 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-22T03:37:31.725 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-22T03:37:31.725 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:37:31.725 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-22T03:37:31.725 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-22T03:37:31.725 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-22T03:37:31.725 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-22T03:37:31.725 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-22T03:37:31.725 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:37:31.725 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:37:32.072 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2024-09-22T03:37:32.165 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-22T03:37:32.166 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-22T03:37:32.166 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-22T03:37:32.166 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-22T03:37:32.166 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-22T03:37:32.166 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-22T03:37:32.166 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi003.front.sepia.ceph.com, path=None, version=distro) 2024-09-22T03:37:32.167 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2024-09-22T03:37:32.224 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-22T03:37:32.225 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-22T03:37:32.225 DEBUG:teuthology.orchestra.run.smithi003:> sudo rpm -qi grub2-tools 2024-09-22T03:37:32.266 INFO:teuthology.orchestra.run.smithi003.stdout:Name : grub2-tools 2024-09-22T03:37:32.266 INFO:teuthology.orchestra.run.smithi003.stdout:Epoch : 1 2024-09-22T03:37:32.266 INFO:teuthology.orchestra.run.smithi003.stdout:Version : 2.06 2024-09-22T03:37:32.266 INFO:teuthology.orchestra.run.smithi003.stdout:Release : 82.el9 2024-09-22T03:37:32.266 INFO:teuthology.orchestra.run.smithi003.stdout:Architecture: x86_64 2024-09-22T03:37:32.266 INFO:teuthology.orchestra.run.smithi003.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-22T03:37:32.266 INFO:teuthology.orchestra.run.smithi003.stdout:Group : Unspecified 2024-09-22T03:37:32.266 INFO:teuthology.orchestra.run.smithi003.stdout:Size : 8274670 2024-09-22T03:37:32.266 INFO:teuthology.orchestra.run.smithi003.stdout:License : GPLv3+ 2024-09-22T03:37:32.267 INFO:teuthology.orchestra.run.smithi003.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-22T03:37:32.267 INFO:teuthology.orchestra.run.smithi003.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-22T03:37:32.267 INFO:teuthology.orchestra.run.smithi003.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-22T03:37:32.267 INFO:teuthology.orchestra.run.smithi003.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-22T03:37:32.267 INFO:teuthology.orchestra.run.smithi003.stdout:Packager : builder@centos.org 2024-09-22T03:37:32.267 INFO:teuthology.orchestra.run.smithi003.stdout:Vendor : CentOS 2024-09-22T03:37:32.267 INFO:teuthology.orchestra.run.smithi003.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-22T03:37:32.267 INFO:teuthology.orchestra.run.smithi003.stdout:Summary : Support tools for GRUB. 2024-09-22T03:37:32.267 INFO:teuthology.orchestra.run.smithi003.stdout:Description : 2024-09-22T03:37:32.267 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:37:32.267 INFO:teuthology.orchestra.run.smithi003.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-22T03:37:32.267 INFO:teuthology.orchestra.run.smithi003.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-22T03:37:32.267 INFO:teuthology.orchestra.run.smithi003.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-22T03:37:32.267 INFO:teuthology.orchestra.run.smithi003.stdout:hardware devices. 2024-09-22T03:37:32.267 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:37:32.267 INFO:teuthology.orchestra.run.smithi003.stdout:This subpackage provides tools for support of all platforms. 2024-09-22T03:37:32.268 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-22T03:37:32.268 INFO:teuthology.task.kernel:Updating grub on smithi003 to boot 5.14.0-508.el9.x86_64 2024-09-22T03:37:32.268 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-22T03:37:32.957 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2024-09-22T03:37:34.295 INFO:teuthology.orchestra.run.smithi003.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-22T03:37:34.325 INFO:teuthology.orchestra.run.smithi003.stderr:done 2024-09-22T03:37:34.327 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-09-22T03:37:34.341 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.RKch34cJCf 2024-09-22T03:37:34.342 DEBUG:teuthology.orchestra.run.smithi003:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.RKch34cJCf 2024-09-22T03:37:34.415 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.RKch34cJCf 2024-09-22T03:37:34.563 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.RKch34cJCf is 6KB 2024-09-22T03:37:34.611 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.RKch34cJCf 2024-09-22T03:37:34.626 DEBUG:teuthology.orchestra.run.smithi003:> sudo /bin/ls /boot/loader/entries || true 2024-09-22T03:37:34.691 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-22T03:37:34.691 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-22T03:37:34.692 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-22T03:37:34.692 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-22T03:37:34.693 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-22T03:37:34.842 DEBUG:teuthology.orchestra.run.smithi003:> sudo shutdown -r now 2024-09-22T03:37:55.010 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-09-22T03:37:55.011 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-22T03:37:55.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:38:04.853 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-22T03:38:04.853 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-22T03:38:04.854 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:38:23.343 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.3 2024-09-22T03:38:32.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-22T03:38:32.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:38:55.035 DEBUG:teuthology.orchestra.remote:timed out 2024-09-22T03:39:04.042 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-22T03:39:04.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:39:04.605 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-09-22T03:39:05.017 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-22T03:39:05.017 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-508.el9.x86_64"... 2024-09-22T03:39:05.017 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-09-22T03:39:05.032 INFO:teuthology.orchestra.run.smithi123.stdout:5.14.0-508.el9.x86_64 2024-09-22T03:39:05.032 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-22T03:39:05.032 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-22T03:39:05.032 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-22T03:39:06.033 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-09-22T03:39:06.033 DEBUG:teuthology.orchestra.run.smithi123:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-22T03:39:06.102 INFO:teuthology.orchestra.run.smithi123.stdout:ttyS1 2024-09-22T03:39:06.124 DEBUG:teuthology.parallel:result is None 2024-09-22T03:39:32.355 DEBUG:teuthology.orchestra.remote:timed out 2024-09-22T03:39:44.357 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-22T03:39:44.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:39:44.719 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-09-22T03:39:45.116 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-22T03:39:45.116 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-22T03:39:45.116 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-09-22T03:39:45.133 INFO:teuthology.orchestra.run.smithi003.stdout:5.14.0-508.el9.x86_64 2024-09-22T03:39:45.133 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-22T03:39:45.133 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-22T03:39:45.133 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-22T03:39:46.134 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-22T03:39:46.134 DEBUG:teuthology.orchestra.run.smithi003:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-22T03:39:46.216 INFO:teuthology.orchestra.run.smithi003.stdout:ttyS1 2024-09-22T03:39:46.237 DEBUG:teuthology.parallel:result is None 2024-09-22T03:39:46.237 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-22T03:39:46.244 INFO:teuthology.task.internal:Creating test directory... 2024-09-22T03:39:46.244 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-22T03:39:46.247 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-22T03:39:46.268 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-22T03:39:46.274 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-22T03:39:46.282 INFO:teuthology.task.internal:Creating archive directory... 2024-09-22T03:39:46.283 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-22T03:39:46.306 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-22T03:39:46.336 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-22T03:39:46.342 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-22T03:39:46.342 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-22T03:39:46.369 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-22T03:39:46.403 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T03:39:46.408 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T03:39:46.416 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T03:39:46.421 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T03:39:46.423 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-22T03:39:46.430 INFO:teuthology.task.internal:Configuring sudo... 2024-09-22T03:39:46.430 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-22T03:39:46.459 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-22T03:39:46.492 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-22T03:39:46.500 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-22T03:39:46.501 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-22T03:39:46.530 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-22T03:39:46.551 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-22T03:39:46.628 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-22T03:39:46.684 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-22T03:39:46.769 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-22T03:39:46.812 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:39:46.812 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-22T03:39:46.879 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-22T03:39:46.924 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-22T03:39:46.974 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-22T03:39:47.060 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-22T03:39:47.105 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-22T03:39:47.105 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-22T03:39:47.175 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2024-09-22T03:39:47.177 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-09-22T03:39:47.220 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-22T03:39:47.271 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-22T03:39:47.742 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-22T03:39:47.752 INFO:teuthology.task.internal:Starting timer... 2024-09-22T03:39:47.753 INFO:teuthology.run_tasks:Running task pcp... 2024-09-22T03:39:47.765 INFO:teuthology.run_tasks:Running task selinux... 2024-09-22T03:39:47.773 DEBUG:teuthology.orchestra.run.smithi003:> sudo service auditd rotate 2024-09-22T03:39:47.835 INFO:teuthology.orchestra.run.smithi003.stdout:Rotating logs: 2024-09-22T03:39:47.836 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-09-22T03:39:47.894 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-09-22T03:39:47.895 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-22T03:39:47.896 DEBUG:teuthology.orchestra.run.smithi003:> /usr/sbin/getenforce 2024-09-22T03:39:47.923 INFO:teuthology.orchestra.run.smithi003.stdout:Permissive 2024-09-22T03:39:47.923 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-09-22T03:39:47.969 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-09-22T03:39:47.969 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi003.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive'} 2024-09-22T03:39:47.970 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-22T03:39:47.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:39:47.998 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-22T03:39:48.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:39:48.025 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-22T03:39:48.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo /usr/sbin/setenforce permissive 2024-09-22T03:39:48.068 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-09-22T03:39:48.097 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-22T03:39:48.106 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-22T03:39:48.116 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-22T03:39:48.117 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi123.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-22T03:43:35.284 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com')] 2024-09-22T03:43:35.285 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-22T03:43:35.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:43:35.365 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-09-22T03:43:35.451 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-22T03:43:35.451 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-22T03:43:35.452 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T03:43:35.524 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-09-22T03:43:35.605 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-22T03:43:35.606 INFO:teuthology.run_tasks:Running task clock... 2024-09-22T03:43:35.621 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-22T03:43:35.621 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-22T03:43:35.621 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T03:43:35.624 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-22T03:43:35.624 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T03:43:35.654 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-22T03:43:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-22T03:43:35.675 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-22T03:43:35.689 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-22T03:43:35.697 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: ntpd: command not found 2024-09-22T03:43:35.709 INFO:teuthology.orchestra.run.smithi003.stdout:506 Cannot talk to daemon 2024-09-22T03:43:35.714 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-09-22T03:43:35.724 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-22T03:43:35.725 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-09-22T03:43:35.739 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-22T03:43:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-22T03:43:35.753 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-22T03:43:35.798 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-09-22T03:43:35.801 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-22T03:43:35.801 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-09-22T03:43:35.801 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:43:35.801 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:43:35.801 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:43:35.802 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:43:35.807 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-09-22T03:43:35.810 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-22T03:43:35.810 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-09-22T03:43:35.811 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:43:35.811 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:43:35.811 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:43:35.811 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T03:43:35.811 INFO:teuthology.run_tasks:Running task install... 2024-09-22T03:43:35.819 DEBUG:teuthology.task.install:project ceph 2024-09-22T03:43:35.819 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208'}} 2024-09-22T03:43:35.819 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208'} 2024-09-22T03:43:35.819 INFO:teuthology.task.install:Using flavor: crimson 2024-09-22T03:43:35.824 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-22T03:43:35.824 INFO:teuthology.task.install:extra packages: [] 2024-09-22T03:43:35.824 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208', 'tag': None, 'wait_for_package': False} 2024-09-22T03:43:35.825 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=62da54b4b83a96ca9976b7e1d0c8272a564a2208 2024-09-22T03:43:35.826 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208', 'tag': None, 'wait_for_package': False} 2024-09-22T03:43:35.826 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=62da54b4b83a96ca9976b7e1d0c8272a564a2208 2024-09-22T03:43:35.969 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/62da54b4b83a96ca9976b7e1d0c8272a564a2208/centos/9/flavors/crimson/ 2024-09-22T03:43:35.969 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5092.g62da54b4 2024-09-22T03:43:35.978 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/62da54b4b83a96ca9976b7e1d0c8272a564a2208/centos/9/flavors/crimson/ 2024-09-22T03:43:35.978 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5092.g62da54b4 2024-09-22T03:43:36.075 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/62da54b4b83a96ca9976b7e1d0c8272a564a2208/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/62da54b4b83a96ca9976b7e1d0c8272a564a2208/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/62da54b4b83a96ca9976b7e1d0c8272a564a2208/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-22T03:43:36.076 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:43:36.076 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-22T03:43:36.085 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/62da54b4b83a96ca9976b7e1d0c8272a564a2208/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/62da54b4b83a96ca9976b7e1d0c8272a564a2208/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/62da54b4b83a96ca9976b7e1d0c8272a564a2208/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-22T03:43:36.086 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-22T03:43:36.086 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-22T03:43:36.102 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-22T03:43:36.103 DEBUG:teuthology.orchestra.run.smithi003:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/62da54b4b83a96ca9976b7e1d0c8272a564a2208/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-22T03:43:36.112 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-22T03:43:36.112 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/62da54b4b83a96ca9976b7e1d0c8272a564a2208/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-22T03:43:36.170 DEBUG:teuthology.orchestra.run.smithi003:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-22T03:43:36.177 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-22T03:43:36.246 DEBUG:teuthology.orchestra.run.smithi003:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-22T03:43:36.252 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-22T03:43:36.311 INFO:teuthology.orchestra.run.smithi003.stdout:check_obsoletes = 1 2024-09-22T03:43:36.312 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum clean all 2024-09-22T03:43:36.316 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-09-22T03:43:36.318 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-09-22T03:43:36.547 INFO:teuthology.orchestra.run.smithi123.stdout:45 files removed 2024-09-22T03:43:36.549 INFO:teuthology.orchestra.run.smithi003.stdout:45 files removed 2024-09-22T03:43:36.573 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw 2024-09-22T03:43:36.577 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-radosgw 2024-09-22T03:43:37.119 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 335 kB/s | 88 kB 00:00 2024-09-22T03:43:37.158 INFO:teuthology.orchestra.run.smithi003.stdout:ceph packages for x86_64 314 kB/s | 88 kB 00:00 2024-09-22T03:43:37.411 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-22T03:43:37.442 INFO:teuthology.orchestra.run.smithi003.stdout:ceph noarch packages 102 kB/s | 24 kB 00:00 2024-09-22T03:43:37.679 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 9.1 kB/s | 2.1 kB 00:00 2024-09-22T03:43:37.709 INFO:teuthology.orchestra.run.smithi003.stdout:ceph source packages 9.1 kB/s | 2.1 kB 00:00 2024-09-22T03:43:38.617 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 9.5 MB/s | 8.3 MB 00:00 2024-09-22T03:43:38.620 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.3 MB 00:00 2024-09-22T03:43:40.480 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-22T03:43:43.427 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 5.5 MB/s | 20 MB 00:03 2024-09-22T03:43:46.514 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.5 MB 00:02 2024-09-22T03:43:48.015 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 128 kB/s | 19 kB 00:00 2024-09-22T03:43:48.719 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 6.5 MB 00:01 2024-09-22T03:43:49.193 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-22T03:43:50.245 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 83 kB/s | 19 kB 00:00 2024-09-22T03:43:51.532 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-22T03:43:54.624 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-22T03:43:56.842 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:43:56.850 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:43:56.850 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-22T03:43:56.850 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:43:56.850 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-22T03:43:56.851 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 10 M 2024-09-22T03:43:56.851 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-09-22T03:43:56.851 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 3.5 M 2024-09-22T03:43:56.851 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 3.8 M 2024-09-22T03:43:56.851 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-22T03:43:56.851 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-22T03:43:56.851 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 6.1 M 2024-09-22T03:43:56.851 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 20 M 2024-09-22T03:43:56.851 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 25 k 2024-09-22T03:43:56.851 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-22T03:43:56.851 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-22T03:43:56.851 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-22T03:43:56.851 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 808 k 2024-09-22T03:43:56.852 INFO:teuthology.orchestra.run.smithi123.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-22T03:43:56.852 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-22T03:43:56.852 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-22T03:43:56.852 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-22T03:43:56.852 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 558 k 2024-09-22T03:43:56.852 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-22T03:43:56.852 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 5.4 M 2024-09-22T03:43:56.852 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-22T03:43:56.852 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-22T03:43:56.852 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-22T03:43:56.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 45 k 2024-09-22T03:43:56.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 148 k 2024-09-22T03:43:56.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 159 k 2024-09-22T03:43:56.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-22T03:43:56.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 314 k 2024-09-22T03:43:56.853 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 302 k 2024-09-22T03:43:56.853 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 99 k 2024-09-22T03:43:56.853 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-22T03:43:56.853 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-22T03:43:56.853 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-22T03:43:56.853 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-22T03:43:56.853 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-09-22T03:43:56.853 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-22T03:43:56.853 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:43:56.853 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-22T03:43:56.853 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:43:56.853 INFO:teuthology.orchestra.run.smithi123.stdout:Install 31 Packages 2024-09-22T03:43:56.853 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 2 Packages 2024-09-22T03:43:56.853 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:43:56.854 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 60 M 2024-09-22T03:43:56.854 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-22T03:43:57.146 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-09-22T03:43:57.401 INFO:teuthology.orchestra.run.smithi123.stdout:(1/33): ceph-base-19.3.0-5092.g62da54b4.el9.x86 14 MB/s | 6.1 MB 00:00 2024-09-22T03:43:57.451 INFO:teuthology.orchestra.run.smithi123.stdout:(2/33): ceph-selinux-19.3.0-5092.g62da54b4.el9. 506 kB/s | 25 kB 00:00 2024-09-22T03:43:57.534 INFO:teuthology.orchestra.run.smithi123.stdout:(3/33): libcephfs2-19.3.0-5092.g62da54b4.el9.x8 9.5 MB/s | 808 kB 00:00 2024-09-22T03:43:57.626 INFO:teuthology.orchestra.run.smithi123.stdout:(4/33): libradosstriper1-19.3.0-5092.g62da54b4. 6.0 MB/s | 558 kB 00:00 2024-09-22T03:43:57.918 INFO:teuthology.orchestra.run.smithi123.stdout:(5/33): ceph-radosgw-19.3.0-5092.g62da54b4.el9. 11 MB/s | 10 MB 00:00 2024-09-22T03:43:58.018 INFO:teuthology.orchestra.run.smithi123.stdout:(6/33): librgw2-19.3.0-5092.g62da54b4.el9.x86_6 14 MB/s | 5.4 MB 00:00 2024-09-22T03:43:58.043 INFO:teuthology.orchestra.run.smithi123.stdout:(7/33): python3-ceph-argparse-19.3.0-5092.g62da 360 kB/s | 45 kB 00:00 2024-09-22T03:43:58.069 INFO:teuthology.orchestra.run.smithi123.stdout:(8/33): python3-ceph-common-19.3.0-5092.g62da54 2.9 MB/s | 148 kB 00:00 2024-09-22T03:43:58.094 INFO:teuthology.orchestra.run.smithi123.stdout:(9/33): python3-cephfs-19.3.0-5092.g62da54b4.el 3.1 MB/s | 159 kB 00:00 2024-09-22T03:43:58.119 INFO:teuthology.orchestra.run.smithi123.stdout:(10/33): python3-rados-19.3.0-5092.g62da54b4.el 6.1 MB/s | 314 kB 00:00 2024-09-22T03:43:58.303 INFO:teuthology.orchestra.run.smithi123.stdout:(11/33): ceph-common-19.3.0-5092.g62da54b4.el9. 15 MB/s | 20 MB 00:01 2024-09-22T03:43:58.328 INFO:teuthology.orchestra.run.smithi123.stdout:(12/33): python3-rbd-19.3.0-5092.g62da54b4.el9. 1.3 MB/s | 302 kB 00:00 2024-09-22T03:43:58.354 INFO:teuthology.orchestra.run.smithi123.stdout:(13/33): python3-rgw-19.3.0-5092.g62da54b4.el9. 425 kB/s | 99 kB 00:00 2024-09-22T03:43:58.454 INFO:teuthology.orchestra.run.smithi123.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 711 kB/s | 107 kB 00:00 2024-09-22T03:43:58.479 INFO:teuthology.orchestra.run.smithi123.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 1.1 MB/s | 171 kB 00:00 2024-09-22T03:43:58.521 INFO:teuthology.orchestra.run.smithi123.stdout:(16/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 959 kB/s | 160 kB 00:00 2024-09-22T03:43:58.546 INFO:teuthology.orchestra.run.smithi123.stdout:(17/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 491 kB/s | 45 kB 00:00 2024-09-22T03:43:58.597 INFO:teuthology.orchestra.run.smithi123.stdout:(18/33): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-22T03:43:58.622 INFO:teuthology.orchestra.run.smithi123.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 561 kB/s | 42 kB 00:00 2024-09-22T03:43:58.647 INFO:teuthology.orchestra.run.smithi123.stdout:(20/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-22T03:43:58.672 INFO:teuthology.orchestra.run.smithi123.stdout:(21/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-22T03:43:58.689 INFO:teuthology.orchestra.run.smithi123.stdout:(22/33): qatlib-service-24.02.0-1.el9.x86_64.rp 535 kB/s | 36 kB 00:00 2024-09-22T03:43:58.739 INFO:teuthology.orchestra.run.smithi123.stdout:(23/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 4.5 MB/s | 308 kB 00:00 2024-09-22T03:43:58.756 INFO:teuthology.orchestra.run.smithi123.stdout:(24/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 435 kB/s | 47 kB 00:00 2024-09-22T03:43:58.773 INFO:teuthology.orchestra.run.smithi123.stdout:(25/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 825 kB/s | 26 kB 00:00 2024-09-22T03:43:58.807 INFO:teuthology.orchestra.run.smithi123.stdout:(26/33): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-22T03:43:58.841 INFO:teuthology.orchestra.run.smithi123.stdout:(27/33): liboath-2.6.7-2.el9.x86_64.rpm 580 kB/s | 49 kB 00:00 2024-09-22T03:43:58.891 INFO:teuthology.orchestra.run.smithi123.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 3.8 MB/s | 191 kB 00:00 2024-09-22T03:43:58.924 INFO:teuthology.orchestra.run.smithi123.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.1 MB/s | 839 kB 00:00 2024-09-22T03:43:59.050 INFO:teuthology.orchestra.run.smithi123.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-09-22T03:43:59.133 INFO:teuthology.orchestra.run.smithi123.stdout:(31/33): librados2-19.3.0-5092.g62da54b4.el9.x8 17 MB/s | 3.5 MB 00:00 2024-09-22T03:43:59.200 INFO:teuthology.orchestra.run.smithi123.stdout:(32/33): libarrow-9.0.0-11.el9.x86_64.rpm 8.6 MB/s | 4.4 MB 00:00 2024-09-22T03:43:59.317 INFO:teuthology.orchestra.run.smithi123.stdout:(33/33): librbd1-19.3.0-5092.g62da54b4.el9.x86_ 14 MB/s | 3.8 MB 00:00 2024-09-22T03:43:59.323 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:43:59.323 INFO:teuthology.orchestra.run.smithi123.stdout:Total 24 MB/s | 60 MB 00:02 2024-09-22T03:43:59.437 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-22T03:43:59.442 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:43:59.456 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:43:59.456 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-09-22T03:43:59.456 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:43:59.456 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-22T03:43:59.456 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 10 M 2024-09-22T03:43:59.456 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrading: 2024-09-22T03:43:59.456 INFO:teuthology.orchestra.run.smithi003.stdout: librados2 x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 3.5 M 2024-09-22T03:43:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1 x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 3.8 M 2024-09-22T03:43:59.457 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-09-22T03:43:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-22T03:43:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 6.1 M 2024-09-22T03:43:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 20 M 2024-09-22T03:43:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 25 k 2024-09-22T03:43:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-22T03:43:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-22T03:43:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-22T03:43:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2 x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 808 k 2024-09-22T03:43:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-22T03:43:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-22T03:43:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-22T03:43:59.457 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-22T03:43:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 558 k 2024-09-22T03:43:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-22T03:43:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2 x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 5.4 M 2024-09-22T03:43:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-22T03:43:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-22T03:43:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-22T03:43:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 45 k 2024-09-22T03:43:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 148 k 2024-09-22T03:43:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 159 k 2024-09-22T03:43:59.458 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-22T03:43:59.459 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 314 k 2024-09-22T03:43:59.459 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 302 k 2024-09-22T03:43:59.459 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 99 k 2024-09-22T03:43:59.459 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-22T03:43:59.459 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-22T03:43:59.459 INFO:teuthology.orchestra.run.smithi003.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-22T03:43:59.459 INFO:teuthology.orchestra.run.smithi003.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-22T03:43:59.459 INFO:teuthology.orchestra.run.smithi003.stdout:Installing weak dependencies: 2024-09-22T03:43:59.459 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-22T03:43:59.459 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:43:59.459 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-22T03:43:59.459 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:43:59.459 INFO:teuthology.orchestra.run.smithi003.stdout:Install 31 Packages 2024-09-22T03:43:59.459 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrade 2 Packages 2024-09-22T03:43:59.460 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:43:59.460 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 60 M 2024-09-22T03:43:59.460 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-22T03:43:59.473 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-22T03:43:59.473 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-22T03:44:00.149 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-22T03:44:00.149 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-22T03:44:00.497 INFO:teuthology.orchestra.run.smithi003.stdout:(1/33): ceph-base-19.3.0-5092.g62da54b4.el9.x86 11 MB/s | 6.1 MB 00:00 2024-09-22T03:44:00.556 INFO:teuthology.orchestra.run.smithi003.stdout:(2/33): ceph-selinux-19.3.0-5092.g62da54b4.el9. 431 kB/s | 25 kB 00:00 2024-09-22T03:44:00.698 INFO:teuthology.orchestra.run.smithi003.stdout:(3/33): ceph-radosgw-19.3.0-5092.g62da54b4.el9. 13 MB/s | 10 MB 00:00 2024-09-22T03:44:00.731 INFO:teuthology.orchestra.run.smithi003.stdout:(4/33): libcephfs2-19.3.0-5092.g62da54b4.el9.x8 4.5 MB/s | 808 kB 00:00 2024-09-22T03:44:00.773 INFO:teuthology.orchestra.run.smithi003.stdout:(5/33): libradosstriper1-19.3.0-5092.g62da54b4. 7.3 MB/s | 558 kB 00:00 2024-09-22T03:44:00.840 INFO:teuthology.orchestra.run.smithi003.stdout:(6/33): python3-ceph-argparse-19.3.0-5092.g62da 677 kB/s | 45 kB 00:00 2024-09-22T03:44:00.924 INFO:teuthology.orchestra.run.smithi003.stdout:(7/33): python3-ceph-common-19.3.0-5092.g62da54 1.7 MB/s | 148 kB 00:00 2024-09-22T03:44:01.141 INFO:teuthology.orchestra.run.smithi003.stdout:(8/33): ceph-common-19.3.0-5092.g62da54b4.el9.x 16 MB/s | 20 MB 00:01 2024-09-22T03:44:01.166 INFO:teuthology.orchestra.run.smithi003.stdout:(9/33): python3-cephfs-19.3.0-5092.g62da54b4.el 659 kB/s | 159 kB 00:00 2024-09-22T03:44:01.192 INFO:teuthology.orchestra.run.smithi003.stdout:(10/33): python3-rados-19.3.0-5092.g62da54b4.el 6.1 MB/s | 314 kB 00:00 2024-09-22T03:44:01.275 INFO:teuthology.orchestra.run.smithi003.stdout:(11/33): librgw2-19.3.0-5092.g62da54b4.el9.x86_ 9.9 MB/s | 5.4 MB 00:00 2024-09-22T03:44:01.300 INFO:teuthology.orchestra.run.smithi003.stdout:(12/33): python3-rgw-19.3.0-5092.g62da54b4.el9. 916 kB/s | 99 kB 00:00 2024-09-22T03:44:01.334 INFO:teuthology.orchestra.run.smithi003.stdout:(13/33): python3-rbd-19.3.0-5092.g62da54b4.el9. 1.8 MB/s | 302 kB 00:00 2024-09-22T03:44:01.401 INFO:teuthology.orchestra.run.smithi003.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 850 kB/s | 107 kB 00:00 2024-09-22T03:44:01.443 INFO:teuthology.orchestra.run.smithi003.stdout:(15/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-22T03:44:01.468 INFO:teuthology.orchestra.run.smithi003.stdout:(16/33): libnbd-1.20.2-2.el9.x86_64.rpm 1.0 MB/s | 171 kB 00:00 2024-09-22T03:44:01.493 INFO:teuthology.orchestra.run.smithi003.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-22T03:44:01.544 INFO:teuthology.orchestra.run.smithi003.stdout:(18/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-22T03:44:01.569 INFO:teuthology.orchestra.run.smithi003.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 563 kB/s | 42 kB 00:00 2024-09-22T03:44:01.594 INFO:teuthology.orchestra.run.smithi003.stdout:(20/33): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-22T03:44:01.611 INFO:teuthology.orchestra.run.smithi003.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 857 kB/s | 36 kB 00:00 2024-09-22T03:44:01.627 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-22T03:44:01.645 INFO:teuthology.orchestra.run.smithi003.stdout:(22/33): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-22T03:44:01.662 INFO:teuthology.orchestra.run.smithi003.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 706 kB/s | 47 kB 00:00 2024-09-22T03:44:01.720 INFO:teuthology.orchestra.run.smithi003.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-22T03:44:01.737 INFO:teuthology.orchestra.run.smithi003.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 3.3 MB/s | 49 kB 00:00 2024-09-22T03:44:01.754 INFO:teuthology.orchestra.run.smithi003.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-09-22T03:44:01.780 INFO:teuthology.orchestra.run.smithi003.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-22T03:44:01.813 INFO:teuthology.orchestra.run.smithi003.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.7 MB/s | 191 kB 00:00 2024-09-22T03:44:01.880 INFO:teuthology.orchestra.run.smithi003.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-09-22T03:44:02.039 INFO:teuthology.orchestra.run.smithi003.stdout:(30/33): librados2-19.3.0-5092.g62da54b4.el9.x8 22 MB/s | 3.5 MB 00:00 2024-09-22T03:44:02.072 INFO:teuthology.orchestra.run.smithi003.stdout:(31/33): thrift-0.15.0-2.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-09-22T03:44:02.127 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-22T03:44:02.148 INFO:teuthology.orchestra.run.smithi003.stdout:(32/33): libarrow-9.0.0-11.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-09-22T03:44:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.3.0-5092.g62da54b4.el9.x86_64 2/35 2024-09-22T03:44:02.223 INFO:teuthology.orchestra.run.smithi003.stdout:(33/33): librbd1-19.3.0-5092.g62da54b4.el9.x86_ 21 MB/s | 3.8 MB 00:00 2024-09-22T03:44:02.227 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:44:02.228 INFO:teuthology.orchestra.run.smithi003.stdout:Total 22 MB/s | 60 MB 00:02 2024-09-22T03:44:02.285 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.3.0-5092.g62da54b4.el9.x86_64 2/35 2024-09-22T03:44:02.349 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-22T03:44:02.375 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.3.0-5092.g62da54b4.el9.x86_64 3/35 2024-09-22T03:44:02.391 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-22T03:44:02.391 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-22T03:44:02.404 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.3.0-5092.g62da54b4.el9.x86_64 4/35 2024-09-22T03:44:02.467 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.3.0-5092.g62da54b4.el9.x86_64 4/35 2024-09-22T03:44:02.528 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 5/35 2024-09-22T03:44:02.558 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/35 2024-09-22T03:44:02.583 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/35 2024-09-22T03:44:02.606 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 8/35 2024-09-22T03:44:02.632 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 8/35 2024-09-22T03:44:02.834 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 9/35 2024-09-22T03:44:02.864 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 9/35 2024-09-22T03:44:02.899 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/35 2024-09-22T03:44:02.942 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.3.0-5092.g62da54b4.el9. 11/35 2024-09-22T03:44:02.971 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.3.0-5092.g62da54b4.el9.x86_64 12/35 2024-09-22T03:44:03.050 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 13/35 2024-09-22T03:44:03.078 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.3.0-5092.g62da54b4.el9.x86_6 14/35 2024-09-22T03:44:03.102 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-22T03:44:03.104 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-22T03:44:03.147 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5092.g62da54b4.el9.x86_6 14/35 2024-09-22T03:44:03.193 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 15/35 2024-09-22T03:44:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-20.el9.x86_64 16/35 2024-09-22T03:44:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/35 2024-09-22T03:44:03.385 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 18/35 2024-09-22T03:44:03.426 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 19/35 2024-09-22T03:44:03.455 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 20/35 2024-09-22T03:44:03.500 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 21/35 2024-09-22T03:44:03.865 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-22T03:44:03.917 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.3.0-5092.g62da54b4.el9.x86_64 23/35 2024-09-22T03:44:04.008 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.3.0-5092.g62da54b4.el9.x86_64 23/35 2024-09-22T03:44:04.038 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.3.0-5092.g62da54b4.el9.x86_64 24/35 2024-09-22T03:44:04.074 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-22T03:44:04.665 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-22T03:44:04.691 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-22T03:44:04.694 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.3.0-5092.g62da54b4.el9.x86_64 27/35 2024-09-22T03:44:04.766 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.3.0-5092.g62da54b4.el9.x86_64 27/35 2024-09-22T03:44:04.848 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.3.0-5092.g62da54b4.el9.x86_64 28/35 2024-09-22T03:44:04.872 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.3.0-5092.g62da54b4.el9.x8 29/35 2024-09-22T03:44:05.232 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-22T03:44:05.308 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librados2-2:19.3.0-5092.g62da54b4.el9.x86_64 2/35 2024-09-22T03:44:05.401 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:19.3.0-5092.g62da54b4.el9.x86_64 2/35 2024-09-22T03:44:05.493 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rados-2:19.3.0-5092.g62da54b4.el9.x86_64 3/35 2024-09-22T03:44:05.518 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs2-2:19.3.0-5092.g62da54b4.el9.x86_64 4/35 2024-09-22T03:44:05.581 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephfs2-2:19.3.0-5092.g62da54b4.el9.x86_64 4/35 2024-09-22T03:44:05.619 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : liboath-2.6.7-2.el9.x86_64 5/35 2024-09-22T03:44:05.656 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/35 2024-09-22T03:44:05.681 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/35 2024-09-22T03:44:05.712 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 8/35 2024-09-22T03:44:05.734 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 8/35 2024-09-22T03:44:05.925 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 9/35 2024-09-22T03:44:05.970 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 9/35 2024-09-22T03:44:06.005 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/35 2024-09-22T03:44:06.050 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-argparse-2:19.3.0-5092.g62da54b4.el9. 11/35 2024-09-22T03:44:06.120 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cephfs-2:19.3.0-5092.g62da54b4.el9.x86_64 12/35 2024-09-22T03:44:06.212 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 13/35 2024-09-22T03:44:06.242 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libradosstriper1-2:19.3.0-5092.g62da54b4.el9.x86_6 14/35 2024-09-22T03:44:06.331 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5092.g62da54b4.el9.x86_6 14/35 2024-09-22T03:44:06.366 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : thrift-0.15.0-2.el9.x86_64 15/35 2024-09-22T03:44:06.479 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : re2-1:20211101-20.el9.x86_64 16/35 2024-09-22T03:44:06.508 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/35 2024-09-22T03:44:06.560 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 18/35 2024-09-22T03:44:06.599 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 19/35 2024-09-22T03:44:06.636 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 20/35 2024-09-22T03:44:06.673 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 21/35 2024-09-22T03:44:07.051 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-22T03:44:07.107 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librbd1-2:19.3.0-5092.g62da54b4.el9.x86_64 23/35 2024-09-22T03:44:07.198 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:19.3.0-5092.g62da54b4.el9.x86_64 23/35 2024-09-22T03:44:07.228 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rbd-2:19.3.0-5092.g62da54b4.el9.x86_64 24/35 2024-09-22T03:44:07.264 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-22T03:44:07.373 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 30/35 2024-09-22T03:44:07.401 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 30/35 2024-09-22T03:44:07.874 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-22T03:44:07.901 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librgw2-2:19.3.0-5092.g62da54b4.el9.x86_64 27/35 2024-09-22T03:44:07.973 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librgw2-2:19.3.0-5092.g62da54b4.el9.x86_64 27/35 2024-09-22T03:44:08.058 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rgw-2:19.3.0-5092.g62da54b4.el9.x86_64 28/35 2024-09-22T03:44:08.087 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-common-2:19.3.0-5092.g62da54b4.el9.x8 29/35 2024-09-22T03:44:08.101 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 30/35 2024-09-22T03:44:08.146 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.3.0-5092.g62da54b4.el9.x86_64 31/35 2024-09-22T03:44:08.235 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.3.0-5092.g62da54b4.el9.x86_64 31/35 2024-09-22T03:44:08.236 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-22T03:44:08.236 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-22T03:44:08.236 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:44:08.272 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.3.0-5092.g62da54b4.el9.x86_64 32/35 2024-09-22T03:44:10.668 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 30/35 2024-09-22T03:44:10.699 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 30/35 2024-09-22T03:44:11.454 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 30/35 2024-09-22T03:44:11.494 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-base-2:19.3.0-5092.g62da54b4.el9.x86_64 31/35 2024-09-22T03:44:11.586 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-base-2:19.3.0-5092.g62da54b4.el9.x86_64 31/35 2024-09-22T03:44:11.586 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-22T03:44:11.586 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-22T03:44:11.586 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:44:11.629 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-selinux-2:19.3.0-5092.g62da54b4.el9.x86_64 32/35 2024-09-22T03:44:18.406 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5092.g62da54b4.el9.x86_64 32/35 2024-09-22T03:44:18.408 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-09-22T03:44:18.408 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-09-22T03:44:18.408 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-09-22T03:44:18.408 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-09-22T03:44:18.408 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-09-22T03:44:18.408 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-09-22T03:44:18.408 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-09-22T03:44:18.408 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:44:19.537 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.3.0-5092.g62da54b4.el9.x86_64 33/35 2024-09-22T03:44:19.563 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5092.g62da54b4.el9.x86_64 33/35 2024-09-22T03:44:19.563 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:44:19.563 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-22T03:44:19.563 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-22T03:44:19.563 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-22T03:44:19.563 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:44:19.565 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-22T03:44:19.648 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-22T03:44:19.648 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-22T03:44:20.933 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-22T03:44:20.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.3.0-5092.g62da54b4.el9.x86_64 1/35 2024-09-22T03:44:20.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 2/35 2024-09-22T03:44:20.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.3.0-5092.g62da54b4.el9.x86_64 3/35 2024-09-22T03:44:20.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.3.0-5092.g62da54b4.el9.x86_64 4/35 2024-09-22T03:44:20.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.3.0-5092.g62da54b4.el9.x86_64 5/35 2024-09-22T03:44:20.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.3.0-5092.g62da54b4.el9.x86_6 6/35 2024-09-22T03:44:20.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.3.0-5092.g62da54b4.el9.x86_64 7/35 2024-09-22T03:44:20.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5092.g62da54b4.el9. 8/35 2024-09-22T03:44:20.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.3.0-5092.g62da54b4.el9.x8 9/35 2024-09-22T03:44:20.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.3.0-5092.g62da54b4.el9.x86_64 10/35 2024-09-22T03:44:20.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.3.0-5092.g62da54b4.el9.x86_64 11/35 2024-09-22T03:44:20.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.3.0-5092.g62da54b4.el9.x86_64 12/35 2024-09-22T03:44:20.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.3.0-5092.g62da54b4.el9.x86_64 13/35 2024-09-22T03:44:20.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-22T03:44:20.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-22T03:44:20.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-22T03:44:20.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-22T03:44:20.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-22T03:44:20.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-22T03:44:20.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-22T03:44:20.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-22T03:44:20.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-22T03:44:20.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-22T03:44:20.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-22T03:44:20.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-22T03:44:20.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-22T03:44:20.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-22T03:44:20.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-22T03:44:20.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-22T03:44:20.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-22T03:44:20.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-22T03:44:20.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.3.0-5092.g62da54b4.el9.x86_64 32/35 2024-09-22T03:44:20.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-22T03:44:20.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.3.0-5092.g62da54b4.el9.x86_64 34/35 2024-09-22T03:44:21.871 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-22T03:44:21.871 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:44:21.871 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-09-22T03:44:21.871 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:21.872 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:21.872 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-22T03:44:21.872 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-22T03:44:21.872 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:21.872 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:21.872 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:21.872 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:21.873 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-22T03:44:21.873 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-22T03:44:21.873 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-22T03:44:21.873 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:21.873 INFO:teuthology.orchestra.run.smithi123.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-22T03:44:21.873 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-22T03:44:21.873 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-22T03:44:21.873 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-22T03:44:21.874 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:21.874 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-22T03:44:21.874 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:21.874 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-22T03:44:21.874 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-22T03:44:21.874 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-22T03:44:21.874 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:21.874 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:21.875 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:21.875 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-22T03:44:21.875 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:21.875 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:21.875 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:21.875 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-22T03:44:21.875 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-22T03:44:21.875 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-22T03:44:21.875 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-20.el9.x86_64 2024-09-22T03:44:21.875 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-22T03:44:21.875 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:44:21.875 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:44:22.227 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-test 2024-09-22T03:44:22.395 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5092.g62da54b4.el9.x86_64 32/35 2024-09-22T03:44:22.395 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /sys 2024-09-22T03:44:22.396 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /proc 2024-09-22T03:44:22.396 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /mnt 2024-09-22T03:44:22.396 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /var/tmp 2024-09-22T03:44:22.396 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /home 2024-09-22T03:44:22.396 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /root 2024-09-22T03:44:22.396 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /tmp 2024-09-22T03:44:22.396 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:44:22.807 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:28 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:44:22.948 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:44:22.949 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:44:22.949 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-22T03:44:22.949 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:44:22.950 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-22T03:44:22.950 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 81 M 2024-09-22T03:44:22.950 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-22T03:44:22.950 INFO:teuthology.orchestra.run.smithi123.stdout: libasan8 x86_64 14.2.1-1.2.el9 appstream 513 k 2024-09-22T03:44:22.950 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 175 k 2024-09-22T03:44:22.950 INFO:teuthology.orchestra.run.smithi123.stdout: libubsan x86_64 11.5.0-2.el9 appstream 183 k 2024-09-22T03:44:22.951 INFO:teuthology.orchestra.run.smithi123.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2024-09-22T03:44:22.951 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-22T03:44:22.951 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-22T03:44:22.951 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-22T03:44:22.951 INFO:teuthology.orchestra.run.smithi123.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-09-22T03:44:22.951 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:44:22.952 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-22T03:44:22.952 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:44:22.952 INFO:teuthology.orchestra.run.smithi123.stdout:Install 9 Packages 2024-09-22T03:44:22.952 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:44:22.953 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 83 M 2024-09-22T03:44:22.953 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 775 M 2024-09-22T03:44:22.954 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-22T03:44:23.525 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-radosgw-2:19.3.0-5092.g62da54b4.el9.x86_64 33/35 2024-09-22T03:44:23.554 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5092.g62da54b4.el9.x86_64 33/35 2024-09-22T03:44:23.555 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:44:23.555 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-22T03:44:23.555 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-22T03:44:23.555 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-22T03:44:23.555 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:44:23.558 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-22T03:44:23.644 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-22T03:44:23.644 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-22T03:44:23.702 INFO:teuthology.orchestra.run.smithi123.stdout:(1/9): lksctp-tools-1.0.19-2.el9.x86_64.rpm 597 kB/s | 94 kB 00:00 2024-09-22T03:44:23.803 INFO:teuthology.orchestra.run.smithi123.stdout:(2/9): libcephsqlite-19.3.0-5092.g62da54b4.el9. 678 kB/s | 175 kB 00:00 2024-09-22T03:44:23.944 INFO:teuthology.orchestra.run.smithi123.stdout:(3/9): libasan8-14.2.1-1.2.el9.x86_64.rpm 2.1 MB/s | 513 kB 00:00 2024-09-22T03:44:23.970 INFO:teuthology.orchestra.run.smithi123.stdout:(4/9): libubsan-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 183 kB 00:00 2024-09-22T03:44:24.045 INFO:teuthology.orchestra.run.smithi123.stdout:(5/9): socat-1.7.4.1-6.el9.x86_64.rpm 4.0 MB/s | 304 kB 00:00 2024-09-22T03:44:24.095 INFO:teuthology.orchestra.run.smithi123.stdout:(6/9): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-22T03:44:24.137 INFO:teuthology.orchestra.run.smithi123.stdout:(7/9): yaml-cpp-0.6.3-5.el9.x86_64.rpm 2.9 MB/s | 122 kB 00:00 2024-09-22T03:44:24.204 INFO:teuthology.orchestra.run.smithi123.stdout:(8/9): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-09-22T03:44:25.016 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-22T03:44:25.016 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-base-2:19.3.0-5092.g62da54b4.el9.x86_64 1/35 2024-09-22T03:44:25.016 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 2/35 2024-09-22T03:44:25.016 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-radosgw-2:19.3.0-5092.g62da54b4.el9.x86_64 3/35 2024-09-22T03:44:25.016 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-selinux-2:19.3.0-5092.g62da54b4.el9.x86_64 4/35 2024-09-22T03:44:25.016 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs2-2:19.3.0-5092.g62da54b4.el9.x86_64 5/35 2024-09-22T03:44:25.016 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libradosstriper1-2:19.3.0-5092.g62da54b4.el9.x86_6 6/35 2024-09-22T03:44:25.017 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librgw2-2:19.3.0-5092.g62da54b4.el9.x86_64 7/35 2024-09-22T03:44:25.017 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5092.g62da54b4.el9. 8/35 2024-09-22T03:44:25.017 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-common-2:19.3.0-5092.g62da54b4.el9.x8 9/35 2024-09-22T03:44:25.017 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cephfs-2:19.3.0-5092.g62da54b4.el9.x86_64 10/35 2024-09-22T03:44:25.017 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rados-2:19.3.0-5092.g62da54b4.el9.x86_64 11/35 2024-09-22T03:44:25.017 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rbd-2:19.3.0-5092.g62da54b4.el9.x86_64 12/35 2024-09-22T03:44:25.017 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rgw-2:19.3.0-5092.g62da54b4.el9.x86_64 13/35 2024-09-22T03:44:25.017 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-22T03:44:25.017 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-22T03:44:25.017 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-22T03:44:25.017 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-22T03:44:25.017 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-22T03:44:25.017 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-22T03:44:25.017 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-22T03:44:25.018 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-22T03:44:25.018 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-22T03:44:25.018 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-22T03:44:25.018 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-22T03:44:25.018 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-22T03:44:25.018 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-22T03:44:25.018 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-22T03:44:25.019 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-22T03:44:25.019 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-22T03:44:25.019 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-22T03:44:25.019 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-22T03:44:25.019 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:19.3.0-5092.g62da54b4.el9.x86_64 32/35 2024-09-22T03:44:25.019 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-22T03:44:25.019 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:19.3.0-5092.g62da54b4.el9.x86_64 34/35 2024-09-22T03:44:25.948 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-22T03:44:25.948 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:44:25.948 INFO:teuthology.orchestra.run.smithi003.stdout:Upgraded: 2024-09-22T03:44:25.948 INFO:teuthology.orchestra.run.smithi003.stdout: librados2-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:25.948 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:25.948 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-22T03:44:25.948 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-22T03:44:25.948 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:25.948 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:25.949 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:25.949 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:25.949 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-22T03:44:25.949 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-22T03:44:25.949 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-22T03:44:25.949 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:25.949 INFO:teuthology.orchestra.run.smithi003.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-22T03:44:25.949 INFO:teuthology.orchestra.run.smithi003.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-22T03:44:25.949 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-22T03:44:25.949 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-22T03:44:25.949 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:25.949 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-22T03:44:25.949 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:25.950 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-22T03:44:25.950 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-22T03:44:25.950 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-22T03:44:25.950 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:25.950 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:25.950 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:25.950 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-22T03:44:25.950 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:25.950 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:25.950 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:25.950 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-22T03:44:25.950 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-22T03:44:25.951 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-22T03:44:25.951 INFO:teuthology.orchestra.run.smithi003.stdout: re2-1:20211101-20.el9.x86_64 2024-09-22T03:44:25.951 INFO:teuthology.orchestra.run.smithi003.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-22T03:44:25.951 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:44:25.951 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:44:26.029 INFO:teuthology.orchestra.run.smithi123.stdout:(9/9): ceph-test-19.3.0-5092.g62da54b4.el9.x86_ 32 MB/s | 81 MB 00:02 2024-09-22T03:44:26.032 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:44:26.033 INFO:teuthology.orchestra.run.smithi123.stdout:Total 27 MB/s | 83 MB 00:03 2024-09-22T03:44:26.114 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-22T03:44:26.136 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-22T03:44:26.136 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-22T03:44:26.305 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-test 2024-09-22T03:44:26.732 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-22T03:44:26.732 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-22T03:44:26.902 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:00:29 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:44:27.037 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:44:27.038 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:44:27.038 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-09-22T03:44:27.038 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:44:27.038 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-22T03:44:27.038 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 81 M 2024-09-22T03:44:27.038 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-09-22T03:44:27.038 INFO:teuthology.orchestra.run.smithi003.stdout: libasan8 x86_64 14.2.1-1.2.el9 appstream 513 k 2024-09-22T03:44:27.038 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 175 k 2024-09-22T03:44:27.039 INFO:teuthology.orchestra.run.smithi003.stdout: libubsan x86_64 11.5.0-2.el9 appstream 183 k 2024-09-22T03:44:27.039 INFO:teuthology.orchestra.run.smithi003.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2024-09-22T03:44:27.039 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-22T03:44:27.039 INFO:teuthology.orchestra.run.smithi003.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-22T03:44:27.039 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-22T03:44:27.039 INFO:teuthology.orchestra.run.smithi003.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-09-22T03:44:27.039 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:44:27.039 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-22T03:44:27.039 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:44:27.039 INFO:teuthology.orchestra.run.smithi003.stdout:Install 9 Packages 2024-09-22T03:44:27.039 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:44:27.040 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 83 M 2024-09-22T03:44:27.040 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 775 M 2024-09-22T03:44:27.040 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-22T03:44:27.417 INFO:teuthology.orchestra.run.smithi003.stdout:(1/9): lksctp-tools-1.0.19-2.el9.x86_64.rpm 517 kB/s | 94 kB 00:00 2024-09-22T03:44:27.443 INFO:teuthology.orchestra.run.smithi003.stdout:(2/9): libcephsqlite-19.3.0-5092.g62da54b4.el9. 844 kB/s | 175 kB 00:00 2024-09-22T03:44:27.576 INFO:teuthology.orchestra.run.smithi003.stdout:(3/9): libubsan-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 183 kB 00:00 2024-09-22T03:44:27.601 INFO:teuthology.orchestra.run.smithi003.stdout:(4/9): libasan8-14.2.1-1.2.el9.x86_64.rpm 2.7 MB/s | 513 kB 00:00 2024-09-22T03:44:27.608 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-22T03:44:27.652 INFO:teuthology.orchestra.run.smithi003.stdout:(5/9): socat-1.7.4.1-6.el9.x86_64.rpm 6.0 MB/s | 304 kB 00:00 2024-09-22T03:44:27.678 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/9 2024-09-22T03:44:27.694 INFO:teuthology.orchestra.run.smithi003.stdout:(6/9): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-09-22T03:44:27.719 INFO:teuthology.orchestra.run.smithi003.stdout:(7/9): xmlstarlet-1.6.1-20.el9.x86_64.rpm 955 kB/s | 64 kB 00:00 2024-09-22T03:44:27.734 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 2/9 2024-09-22T03:44:27.761 INFO:teuthology.orchestra.run.smithi003.stdout:(8/9): yaml-cpp-0.6.3-5.el9.x86_64.rpm 1.8 MB/s | 122 kB 00:00 2024-09-22T03:44:27.783 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 3/9 2024-09-22T03:44:27.810 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 4/9 2024-09-22T03:44:27.841 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libubsan-11.5.0-2.el9.x86_64 5/9 2024-09-22T03:44:27.866 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libasan8-14.2.1-1.2.el9.x86_64 6/9 2024-09-22T03:44:27.918 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libasan8-14.2.1-1.2.el9.x86_64 6/9 2024-09-22T03:44:27.957 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 7/9 2024-09-22T03:44:27.984 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.3.0-5092.g62da54b4.el9.x86_64 8/9 2024-09-22T03:44:29.602 INFO:teuthology.orchestra.run.smithi003.stdout:(9/9): ceph-test-19.3.0-5092.g62da54b4.el9.x86_ 34 MB/s | 81 MB 00:02 2024-09-22T03:44:29.606 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:44:29.606 INFO:teuthology.orchestra.run.smithi003.stdout:Total 32 MB/s | 83 MB 00:02 2024-09-22T03:44:29.692 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-22T03:44:29.718 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-22T03:44:29.718 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-22T03:44:30.352 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-22T03:44:30.352 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-22T03:44:31.268 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-22T03:44:31.357 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/9 2024-09-22T03:44:31.424 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 2/9 2024-09-22T03:44:31.475 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 3/9 2024-09-22T03:44:31.508 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 4/9 2024-09-22T03:44:31.540 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libubsan-11.5.0-2.el9.x86_64 5/9 2024-09-22T03:44:31.572 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libasan8-14.2.1-1.2.el9.x86_64 6/9 2024-09-22T03:44:31.643 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libasan8-14.2.1-1.2.el9.x86_64 6/9 2024-09-22T03:44:31.682 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 7/9 2024-09-22T03:44:31.707 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephsqlite-2:19.3.0-5092.g62da54b4.el9.x86_64 8/9 2024-09-22T03:44:38.865 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5092.g62da54b4.el9.x86_64 8/9 2024-09-22T03:44:38.941 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.3.0-5092.g62da54b4.el9.x86_64 9/9 2024-09-22T03:44:43.210 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5092.g62da54b4.el9.x86_64 8/9 2024-09-22T03:44:43.291 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-test-2:19.3.0-5092.g62da54b4.el9.x86_64 9/9 2024-09-22T03:44:44.698 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-test-2:19.3.0-5092.g62da54b4.el9.x86_64 9/9 2024-09-22T03:44:44.698 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.3.0-5092.g62da54b4.el9.x86_64 1/9 2024-09-22T03:44:44.698 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.3.0-5092.g62da54b4.el9.x86_64 2/9 2024-09-22T03:44:44.698 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 3/9 2024-09-22T03:44:44.698 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libasan8-14.2.1-1.2.el9.x86_64 4/9 2024-09-22T03:44:44.698 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libubsan-11.5.0-2.el9.x86_64 5/9 2024-09-22T03:44:44.699 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 6/9 2024-09-22T03:44:44.699 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 7/9 2024-09-22T03:44:44.699 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 8/9 2024-09-22T03:44:45.098 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 9/9 2024-09-22T03:44:45.098 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:44:45.098 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-22T03:44:45.098 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:45.098 INFO:teuthology.orchestra.run.smithi123.stdout: libasan8-14.2.1-1.2.el9.x86_64 2024-09-22T03:44:45.098 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:45.098 INFO:teuthology.orchestra.run.smithi123.stdout: libubsan-11.5.0-2.el9.x86_64 2024-09-22T03:44:45.098 INFO:teuthology.orchestra.run.smithi123.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-09-22T03:44:45.098 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-22T03:44:45.098 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-22T03:44:45.098 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-22T03:44:45.099 INFO:teuthology.orchestra.run.smithi123.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-09-22T03:44:45.099 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:44:45.099 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:44:45.455 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph 2024-09-22T03:44:46.044 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:52 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:44:46.190 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:44:46.192 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:44:46.193 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-22T03:44:46.193 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:44:46.193 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-22T03:44:46.193 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 6.5 k 2024-09-22T03:44:46.193 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-22T03:44:46.193 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 2.2 M 2024-09-22T03:44:46.193 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 1.6 M 2024-09-22T03:44:46.193 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 292 k 2024-09-22T03:44:46.193 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 5.1 M 2024-09-22T03:44:46.193 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 70 M 2024-09-22T03:44:46.193 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-22T03:44:46.193 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-22T03:44:46.193 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-22T03:44:46.194 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-22T03:44:46.194 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-22T03:44:46.194 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-22T03:44:46.194 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-22T03:44:46.194 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-22T03:44:46.194 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-22T03:44:46.194 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-22T03:44:46.194 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections 2024-09-22T03:44:46.194 INFO:teuthology.orchestra.run.smithi123.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-22T03:44:46.194 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-22T03:44:46.194 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-22T03:44:46.194 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-22T03:44:46.195 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-22T03:44:46.195 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-22T03:44:46.195 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-22T03:44:46.195 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-22T03:44:46.195 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-22T03:44:46.195 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-22T03:44:46.195 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-22T03:44:46.195 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-22T03:44:46.195 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-22T03:44:46.195 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-22T03:44:46.195 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-22T03:44:46.195 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-22T03:44:46.195 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-22T03:44:46.195 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-22T03:44:46.195 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-22T03:44:46.196 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-22T03:44:46.196 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-22T03:44:46.196 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:44:46.196 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-22T03:44:46.196 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:44:46.196 INFO:teuthology.orchestra.run.smithi123.stdout:Install 37 Packages 2024-09-22T03:44:46.196 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:44:46.196 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 84 M 2024-09-22T03:44:46.196 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 896 M 2024-09-22T03:44:46.197 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-22T03:44:46.549 INFO:teuthology.orchestra.run.smithi123.stdout:(1/37): ceph-19.3.0-5092.g62da54b4.el9.x86_64.r 50 kB/s | 6.5 kB 00:00 2024-09-22T03:44:46.742 INFO:teuthology.orchestra.run.smithi123.stdout:(2/37): ceph-mgr-19.3.0-5092.g62da54b4.el9.x86_ 4.9 MB/s | 1.6 MB 00:00 2024-09-22T03:44:46.792 INFO:teuthology.orchestra.run.smithi123.stdout:(3/37): ceph-mds-19.3.0-5092.g62da54b4.el9.x86_ 5.9 MB/s | 2.2 MB 00:00 2024-09-22T03:44:46.892 INFO:teuthology.orchestra.run.smithi123.stdout:(4/37): ceph-mgr-modules-core-19.3.0-5092.g62da 2.9 MB/s | 292 kB 00:00 2024-09-22T03:44:46.970 INFO:teuthology.orchestra.run.smithi123.stdout:(5/37): ceph-mon-19.3.0-5092.g62da54b4.el9.x86_ 12 MB/s | 5.1 MB 00:00 2024-09-22T03:44:47.087 INFO:teuthology.orchestra.run.smithi123.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-22T03:44:47.129 INFO:teuthology.orchestra.run.smithi123.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-22T03:44:47.171 INFO:teuthology.orchestra.run.smithi123.stdout:(8/37): python3-cryptography-36.0.1-4.el9.x86_6 6.1 MB/s | 1.2 MB 00:00 2024-09-22T03:44:47.196 INFO:teuthology.orchestra.run.smithi123.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-09-22T03:44:47.222 INFO:teuthology.orchestra.run.smithi123.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-09-22T03:44:47.247 INFO:teuthology.orchestra.run.smithi123.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-22T03:44:47.764 INFO:teuthology.orchestra.run.smithi123.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 348 kB/s | 188 kB 00:00 2024-09-22T03:44:47.797 INFO:teuthology.orchestra.run.smithi123.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 313 kB/s | 172 kB 00:00 2024-09-22T03:44:47.881 INFO:teuthology.orchestra.run.smithi123.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 298 kB/s | 35 kB 00:00 2024-09-22T03:44:47.906 INFO:teuthology.orchestra.run.smithi123.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 710 kB/s | 77 kB 00:00 2024-09-22T03:44:47.998 INFO:teuthology.orchestra.run.smithi123.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 436 kB/s | 51 kB 00:00 2024-09-22T03:44:48.015 INFO:teuthology.orchestra.run.smithi123.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-22T03:44:48.032 INFO:teuthology.orchestra.run.smithi123.stdout:(18/37): lua-devel-5.4.4-4.el9.x86_64.rpm 667 kB/s | 22 kB 00:00 2024-09-22T03:44:48.090 INFO:teuthology.orchestra.run.smithi123.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 745 kB/s | 43 kB 00:00 2024-09-22T03:44:48.124 INFO:teuthology.orchestra.run.smithi123.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-09-22T03:44:48.775 INFO:teuthology.orchestra.run.smithi123.stdout:(21/37): ceph-osd-19.3.0-5092.g62da54b4.el9.x86 35 MB/s | 70 MB 00:02 2024-09-22T03:44:48.800 INFO:teuthology.orchestra.run.smithi123.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 242 kB/s | 172 kB 00:00 2024-09-22T03:44:48.834 INFO:teuthology.orchestra.run.smithi123.stdout:(23/37): python3-cherrypy-18.6.1-2.el9.noarch.r 506 kB/s | 358 kB 00:00 2024-09-22T03:44:48.859 INFO:teuthology.orchestra.run.smithi123.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 346 kB/s | 18 kB 00:00 2024-09-22T03:44:48.884 INFO:teuthology.orchestra.run.smithi123.stdout:(25/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 98 kB/s | 11 kB 00:00 2024-09-22T03:44:48.909 INFO:teuthology.orchestra.run.smithi123.stdout:(26/37): python3-jaraco-collections-3.0.0-8.el9 315 kB/s | 23 kB 00:00 2024-09-22T03:44:48.934 INFO:teuthology.orchestra.run.smithi123.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-09-22T03:44:48.960 INFO:teuthology.orchestra.run.smithi123.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 260 kB/s | 20 kB 00:00 2024-09-22T03:44:48.985 INFO:teuthology.orchestra.run.smithi123.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 613 kB/s | 46 kB 00:00 2024-09-22T03:44:49.010 INFO:teuthology.orchestra.run.smithi123.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-22T03:44:49.035 INFO:teuthology.orchestra.run.smithi123.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-22T03:44:49.061 INFO:teuthology.orchestra.run.smithi123.stdout:(32/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-22T03:44:49.086 INFO:teuthology.orchestra.run.smithi123.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 713 kB/s | 36 kB 00:00 2024-09-22T03:44:49.111 INFO:teuthology.orchestra.run.smithi123.stdout:(34/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 892 kB/s | 90 kB 00:00 2024-09-22T03:44:49.136 INFO:teuthology.orchestra.run.smithi123.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 806 kB/s | 20 kB 00:00 2024-09-22T03:44:49.146 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-test-2:19.3.0-5092.g62da54b4.el9.x86_64 9/9 2024-09-22T03:44:49.146 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-test-2:19.3.0-5092.g62da54b4.el9.x86_64 1/9 2024-09-22T03:44:49.147 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephsqlite-2:19.3.0-5092.g62da54b4.el9.x86_64 2/9 2024-09-22T03:44:49.147 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 3/9 2024-09-22T03:44:49.147 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libasan8-14.2.1-1.2.el9.x86_64 4/9 2024-09-22T03:44:49.147 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libubsan-11.5.0-2.el9.x86_64 5/9 2024-09-22T03:44:49.147 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 6/9 2024-09-22T03:44:49.147 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 7/9 2024-09-22T03:44:49.147 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 8/9 2024-09-22T03:44:49.162 INFO:teuthology.orchestra.run.smithi123.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-22T03:44:49.187 INFO:teuthology.orchestra.run.smithi123.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-22T03:44:49.194 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:44:49.194 INFO:teuthology.orchestra.run.smithi123.stdout:Total 28 MB/s | 84 MB 00:02 2024-09-22T03:44:49.343 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-22T03:44:49.385 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-22T03:44:49.385 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-22T03:44:49.526 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 9/9 2024-09-22T03:44:49.526 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:44:49.526 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-22T03:44:49.526 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:49.526 INFO:teuthology.orchestra.run.smithi003.stdout: libasan8-14.2.1-1.2.el9.x86_64 2024-09-22T03:44:49.526 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:44:49.527 INFO:teuthology.orchestra.run.smithi003.stdout: libubsan-11.5.0-2.el9.x86_64 2024-09-22T03:44:49.527 INFO:teuthology.orchestra.run.smithi003.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-09-22T03:44:49.527 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-22T03:44:49.527 INFO:teuthology.orchestra.run.smithi003.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-22T03:44:49.527 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-22T03:44:49.527 INFO:teuthology.orchestra.run.smithi003.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-09-22T03:44:49.527 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:44:49.527 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:44:49.845 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph 2024-09-22T03:44:50.177 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-22T03:44:50.177 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-22T03:44:50.446 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:00:53 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:44:50.599 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:44:50.602 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:44:50.602 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-09-22T03:44:50.602 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:44:50.602 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-22T03:44:50.602 INFO:teuthology.orchestra.run.smithi003.stdout: ceph x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 6.5 k 2024-09-22T03:44:50.602 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-09-22T03:44:50.602 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 2.2 M 2024-09-22T03:44:50.602 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 1.6 M 2024-09-22T03:44:50.602 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 292 k 2024-09-22T03:44:50.602 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 5.1 M 2024-09-22T03:44:50.603 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 70 M 2024-09-22T03:44:50.603 INFO:teuthology.orchestra.run.smithi003.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-22T03:44:50.603 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-22T03:44:50.603 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-22T03:44:50.603 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-22T03:44:50.603 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-22T03:44:50.603 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-22T03:44:50.603 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-22T03:44:50.603 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-22T03:44:50.603 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-22T03:44:50.603 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-22T03:44:50.603 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections 2024-09-22T03:44:50.603 INFO:teuthology.orchestra.run.smithi003.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-22T03:44:50.603 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-22T03:44:50.604 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-22T03:44:50.604 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-22T03:44:50.604 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-22T03:44:50.604 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-22T03:44:50.604 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-22T03:44:50.604 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-22T03:44:50.604 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-22T03:44:50.604 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-22T03:44:50.604 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-22T03:44:50.604 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-22T03:44:50.604 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-22T03:44:50.604 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-22T03:44:50.605 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-22T03:44:50.605 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-22T03:44:50.605 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-22T03:44:50.605 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-22T03:44:50.605 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-22T03:44:50.605 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-22T03:44:50.605 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-22T03:44:50.605 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:44:50.605 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-22T03:44:50.605 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:44:50.605 INFO:teuthology.orchestra.run.smithi003.stdout:Install 37 Packages 2024-09-22T03:44:50.605 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:44:50.606 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 84 M 2024-09-22T03:44:50.606 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 896 M 2024-09-22T03:44:50.606 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-22T03:44:51.018 INFO:teuthology.orchestra.run.smithi003.stdout:(1/37): ceph-19.3.0-5092.g62da54b4.el9.x86_64.r 43 kB/s | 6.5 kB 00:00 2024-09-22T03:44:51.352 INFO:teuthology.orchestra.run.smithi003.stdout:(2/37): ceph-mgr-19.3.0-5092.g62da54b4.el9.x86_ 3.3 MB/s | 1.6 MB 00:00 2024-09-22T03:44:51.485 INFO:teuthology.orchestra.run.smithi003.stdout:(3/37): ceph-mds-19.3.0-5092.g62da54b4.el9.x86_ 3.6 MB/s | 2.2 MB 00:00 2024-09-22T03:44:51.552 INFO:teuthology.orchestra.run.smithi003.stdout:(4/37): ceph-mgr-modules-core-19.3.0-5092.g62da 4.3 MB/s | 292 kB 00:00 2024-09-22T03:44:51.811 INFO:teuthology.orchestra.run.smithi003.stdout:(5/37): ceph-mon-19.3.0-5092.g62da54b4.el9.x86_ 6.5 MB/s | 5.1 MB 00:00 2024-09-22T03:44:51.836 INFO:teuthology.orchestra.run.smithi003.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 893 kB/s | 253 kB 00:00 2024-09-22T03:44:51.841 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-22T03:44:51.887 INFO:teuthology.orchestra.run.smithi003.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-22T03:44:51.906 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-22T03:44:51.937 INFO:teuthology.orchestra.run.smithi003.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.6 MB/s | 135 kB 00:00 2024-09-22T03:44:51.955 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-22T03:44:51.983 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-22T03:44:51.987 INFO:teuthology.orchestra.run.smithi003.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-22T03:44:52.046 INFO:teuthology.orchestra.run.smithi003.stdout:(10/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-22T03:44:52.053 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-22T03:44:52.104 INFO:teuthology.orchestra.run.smithi003.stdout:(11/37): python3-cryptography-36.0.1-4.el9.x86_ 4.2 MB/s | 1.2 MB 00:00 2024-09-22T03:44:52.107 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-22T03:44:52.139 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-22T03:44:52.233 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-22T03:44:52.280 INFO:teuthology.orchestra.run.smithi003.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 807 kB/s | 188 kB 00:00 2024-09-22T03:44:52.303 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-22T03:44:52.305 INFO:teuthology.orchestra.run.smithi003.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 860 kB/s | 172 kB 00:00 2024-09-22T03:44:52.330 INFO:teuthology.orchestra.run.smithi003.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 692 kB/s | 35 kB 00:00 2024-09-22T03:44:52.355 INFO:teuthology.orchestra.run.smithi003.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-22T03:44:52.359 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-22T03:44:52.381 INFO:teuthology.orchestra.run.smithi003.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-22T03:44:52.402 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-22T03:44:52.414 INFO:teuthology.orchestra.run.smithi003.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 708 kB/s | 42 kB 00:00 2024-09-22T03:44:52.432 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-22T03:44:52.465 INFO:teuthology.orchestra.run.smithi003.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-22T03:44:52.467 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-22T03:44:52.490 INFO:teuthology.orchestra.run.smithi003.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-09-22T03:44:52.500 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-22T03:44:52.515 INFO:teuthology.orchestra.run.smithi003.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 870 kB/s | 43 kB 00:00 2024-09-22T03:44:52.549 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-22T03:44:52.557 INFO:teuthology.orchestra.run.smithi003.stdout:(21/37): python3-cherrypy-18.6.1-2.el9.noarch.r 10 MB/s | 358 kB 00:00 2024-09-22T03:44:52.583 INFO:teuthology.orchestra.run.smithi003.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 466 kB/s | 11 kB 00:00 2024-09-22T03:44:52.584 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-22T03:44:52.608 INFO:teuthology.orchestra.run.smithi003.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 711 kB/s | 18 kB 00:00 2024-09-22T03:44:52.626 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-22T03:44:52.633 INFO:teuthology.orchestra.run.smithi003.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 928 kB/s | 23 kB 00:00 2024-09-22T03:44:52.658 INFO:teuthology.orchestra.run.smithi003.stdout:(25/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-22T03:44:52.674 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-22T03:44:52.684 INFO:teuthology.orchestra.run.smithi003.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-09-22T03:44:52.709 INFO:teuthology.orchestra.run.smithi003.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-22T03:44:52.715 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-22T03:44:52.734 INFO:teuthology.orchestra.run.smithi003.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-09-22T03:44:52.759 INFO:teuthology.orchestra.run.smithi003.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-22T03:44:52.785 INFO:teuthology.orchestra.run.smithi003.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 661 kB/s | 16 kB 00:00 2024-09-22T03:44:52.788 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-22T03:44:52.810 INFO:teuthology.orchestra.run.smithi003.stdout:(31/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.5 MB/s | 90 kB 00:00 2024-09-22T03:44:52.835 INFO:teuthology.orchestra.run.smithi003.stdout:(32/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-22T03:44:52.861 INFO:teuthology.orchestra.run.smithi003.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 711 kB/s | 36 kB 00:00 2024-09-22T03:44:52.903 INFO:teuthology.orchestra.run.smithi003.stdout:(34/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-09-22T03:44:52.919 INFO:teuthology.orchestra.run.smithi003.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2024-09-22T03:44:52.927 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-22T03:44:52.945 INFO:teuthology.orchestra.run.smithi003.stdout:(36/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.0 MB/s | 427 kB 00:00 2024-09-22T03:44:53.006 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-22T03:44:53.059 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-22T03:44:53.097 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-22T03:44:53.146 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-22T03:44:53.210 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-22T03:44:53.398 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-22T03:44:53.443 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-22T03:44:53.521 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-22T03:44:53.609 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-22T03:44:53.659 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-22T03:44:53.788 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-22T03:44:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5092.g62da54b4.el9. 32/37 2024-09-22T03:44:54.003 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.3.0-5092.g62da54b4.el9.x86_64 33/37 2024-09-22T03:44:54.028 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5092.g62da54b4.el9.x86_64 33/37 2024-09-22T03:44:54.029 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:44:54.029 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-22T03:44:54.029 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-22T03:44:54.029 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-22T03:44:54.029 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:44:54.337 INFO:teuthology.orchestra.run.smithi003.stdout:(37/37): ceph-osd-19.3.0-5092.g62da54b4.el9.x86 24 MB/s | 70 MB 00:02 2024-09-22T03:44:54.340 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:44:54.340 INFO:teuthology.orchestra.run.smithi003.stdout:Total 23 MB/s | 84 MB 00:03 2024-09-22T03:44:54.497 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-22T03:44:54.544 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-22T03:44:54.544 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-22T03:44:55.364 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-22T03:44:55.364 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-22T03:44:57.163 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-22T03:44:57.222 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-22T03:44:57.255 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-22T03:44:57.283 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-22T03:44:57.358 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-22T03:44:57.415 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-22T03:44:57.448 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-22T03:44:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-22T03:44:57.632 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-22T03:44:57.687 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-22T03:44:57.721 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-22T03:44:57.751 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-22T03:44:57.787 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-22T03:44:57.819 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-22T03:44:57.860 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-22T03:44:57.903 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-22T03:44:57.945 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-22T03:44:57.995 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-22T03:44:58.034 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-22T03:44:58.132 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-22T03:44:58.287 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-22T03:44:58.370 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-22T03:44:58.422 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-22T03:44:58.468 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-22T03:44:58.534 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-22T03:44:58.599 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-22T03:44:58.794 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-22T03:44:58.847 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-22T03:44:58.936 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-22T03:44:59.034 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-22T03:44:59.080 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-22T03:44:59.224 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-22T03:44:59.433 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5092.g62da54b4.el9. 32/37 2024-09-22T03:44:59.463 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-2:19.3.0-5092.g62da54b4.el9.x86_64 33/37 2024-09-22T03:44:59.491 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5092.g62da54b4.el9.x86_64 33/37 2024-09-22T03:44:59.492 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:44:59.492 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-22T03:44:59.492 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-22T03:44:59.492 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-22T03:44:59.492 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.3.0-5092.g62da54b4.el9.x86_64 34/37 2024-09-22T03:45:04.580 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.3.0-5092.g62da54b4.el9.x86_64 34/37 2024-09-22T03:45:04.581 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:45:04.581 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-22T03:45:04.581 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-22T03:45:04.581 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-22T03:45:04.581 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:05.128 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.3.0-5092.g62da54b4.el9.x86_64 35/37 2024-09-22T03:45:05.152 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.3.0-5092.g62da54b4.el9.x86_64 35/37 2024-09-22T03:45:05.152 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:45:05.152 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-22T03:45:05.152 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-22T03:45:05.152 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-22T03:45:05.152 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:05.395 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.3.0-5092.g62da54b4.el9.x86_64 36/37 2024-09-22T03:45:05.429 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.3.0-5092.g62da54b4.el9.x86_64 36/37 2024-09-22T03:45:05.429 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:45:05.429 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-22T03:45:05.429 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-22T03:45:05.429 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-22T03:45:05.430 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:05.495 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.3.0-5092.g62da54b4.el9.x86_64 37/37 2024-09-22T03:45:10.721 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-osd-2:19.3.0-5092.g62da54b4.el9.x86_64 34/37 2024-09-22T03:45:10.750 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-osd-2:19.3.0-5092.g62da54b4.el9.x86_64 34/37 2024-09-22T03:45:10.751 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:45:10.751 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-22T03:45:10.751 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-22T03:45:10.751 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-22T03:45:10.751 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:11.330 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mon-2:19.3.0-5092.g62da54b4.el9.x86_64 35/37 2024-09-22T03:45:11.355 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mon-2:19.3.0-5092.g62da54b4.el9.x86_64 35/37 2024-09-22T03:45:11.356 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:45:11.356 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-22T03:45:11.356 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-22T03:45:11.356 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-22T03:45:11.356 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:11.606 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mds-2:19.3.0-5092.g62da54b4.el9.x86_64 36/37 2024-09-22T03:45:11.631 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mds-2:19.3.0-5092.g62da54b4.el9.x86_64 36/37 2024-09-22T03:45:11.632 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:45:11.632 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-22T03:45:11.632 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-22T03:45:11.632 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-22T03:45:11.632 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:11.697 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-2:19.3.0-5092.g62da54b4.el9.x86_64 37/37 2024-09-22T03:45:11.764 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-2:19.3.0-5092.g62da54b4.el9.x86_64 37/37 2024-09-22T03:45:11.764 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.3.0-5092.g62da54b4.el9.x86_64 1/37 2024-09-22T03:45:11.764 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.3.0-5092.g62da54b4.el9.x86_64 2/37 2024-09-22T03:45:11.764 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.3.0-5092.g62da54b4.el9.x86_64 3/37 2024-09-22T03:45:11.764 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.3.0-5092.g62da54b4.el9.x86_64 4/37 2024-09-22T03:45:11.765 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.3.0-5092.g62da54b4.el9.x86_64 5/37 2024-09-22T03:45:11.765 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5092.g62da54b4.el9. 6/37 2024-09-22T03:45:11.765 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-22T03:45:11.765 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-22T03:45:11.765 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-22T03:45:11.765 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-22T03:45:11.765 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-22T03:45:11.765 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-22T03:45:11.765 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-22T03:45:11.765 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-22T03:45:11.765 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-22T03:45:11.765 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-22T03:45:11.765 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-22T03:45:11.765 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-22T03:45:11.766 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-22T03:45:11.766 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-22T03:45:11.766 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-22T03:45:11.766 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-22T03:45:11.766 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-22T03:45:11.766 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-22T03:45:11.766 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-22T03:45:11.766 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-22T03:45:11.766 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-22T03:45:11.766 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-22T03:45:11.766 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-22T03:45:11.766 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-22T03:45:11.767 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-22T03:45:11.767 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-22T03:45:11.767 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-22T03:45:11.767 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-22T03:45:11.767 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-22T03:45:11.767 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-22T03:45:12.684 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-22T03:45:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:12.685 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-22T03:45:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:45:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:45:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:45:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:45:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:45:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:45:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-09-22T03:45:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-22T03:45:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-22T03:45:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-22T03:45:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-22T03:45:12.685 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-22T03:45:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-22T03:45:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-22T03:45:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-22T03:45:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-22T03:45:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-22T03:45:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-22T03:45:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-22T03:45:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-22T03:45:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-22T03:45:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-22T03:45:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-22T03:45:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-22T03:45:12.686 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-22T03:45:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-09-22T03:45:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-22T03:45:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-22T03:45:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-22T03:45:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-22T03:45:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-22T03:45:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-22T03:45:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-22T03:45:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-22T03:45:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-22T03:45:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-22T03:45:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-22T03:45:12.687 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:12.687 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:45:12.953 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-base 2024-09-22T03:45:13.531 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:19 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:45:13.620 INFO:teuthology.orchestra.run.smithi123.stdout:Package ceph-base-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:45:13.662 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:45:13.663 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-22T03:45:13.663 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:45:13.723 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install cephadm 2024-09-22T03:45:14.294 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:20 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:45:14.424 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:45:14.424 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:14.425 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-22T03:45:14.425 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:14.425 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-22T03:45:14.425 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 784 k 2024-09-22T03:45:14.425 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:14.425 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-22T03:45:14.425 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:14.425 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-09-22T03:45:14.425 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:14.425 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 784 k 2024-09-22T03:45:14.425 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 791 k 2024-09-22T03:45:14.425 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-22T03:45:14.681 INFO:teuthology.orchestra.run.smithi123.stdout:cephadm-19.3.0-5092.g62da54b4.el9.noarch.rpm 3.0 MB/s | 784 kB 00:00 2024-09-22T03:45:14.681 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:45:14.682 INFO:teuthology.orchestra.run.smithi123.stdout:Total 3.0 MB/s | 784 kB 00:00 2024-09-22T03:45:14.682 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-22T03:45:14.687 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-22T03:45:14.687 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-22T03:45:14.693 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-22T03:45:14.693 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-22T03:45:14.768 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-22T03:45:15.243 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-22T03:45:15.377 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-22T03:45:15.762 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-22T03:45:16.003 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-22T03:45:16.003 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:16.003 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-22T03:45:16.003 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:45:16.003 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:16.003 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:45:16.185 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-immutable-object-cache 2024-09-22T03:45:16.777 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:22 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:45:16.910 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:45:16.911 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:16.911 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repo Size 2024-09-22T03:45:16.911 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:16.911 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-22T03:45:16.911 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 158 k 2024-09-22T03:45:16.911 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:16.911 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-22T03:45:16.911 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:16.911 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-09-22T03:45:16.911 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:16.911 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 158 k 2024-09-22T03:45:16.912 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 644 k 2024-09-22T03:45:16.912 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-22T03:45:17.119 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-immutable-object-cache-19.3.0-5092.g62da54 761 kB/s | 158 kB 00:00 2024-09-22T03:45:17.120 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:45:17.120 INFO:teuthology.orchestra.run.smithi123.stdout:Total 755 kB/s | 158 kB 00:00 2024-09-22T03:45:17.121 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-22T03:45:17.130 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-22T03:45:17.130 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-22T03:45:17.188 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-22T03:45:17.189 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-22T03:45:17.337 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-22T03:45:17.412 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5092.g62da54b4. 1/1 2024-09-22T03:45:17.437 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5092.g62da54b4. 1/1 2024-09-22T03:45:17.437 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:45:17.437 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-22T03:45:17.437 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:18.286 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5092.g62da54b4. 1/1 2024-09-22T03:45:18.286 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:18.286 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-22T03:45:18.286 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:45:18.286 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:18.286 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:45:18.388 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-2:19.3.0-5092.g62da54b4.el9.x86_64 37/37 2024-09-22T03:45:18.388 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-2:19.3.0-5092.g62da54b4.el9.x86_64 1/37 2024-09-22T03:45:18.389 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mds-2:19.3.0-5092.g62da54b4.el9.x86_64 2/37 2024-09-22T03:45:18.389 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-2:19.3.0-5092.g62da54b4.el9.x86_64 3/37 2024-09-22T03:45:18.389 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mon-2:19.3.0-5092.g62da54b4.el9.x86_64 4/37 2024-09-22T03:45:18.389 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-osd-2:19.3.0-5092.g62da54b4.el9.x86_64 5/37 2024-09-22T03:45:18.389 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5092.g62da54b4.el9. 6/37 2024-09-22T03:45:18.389 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-22T03:45:18.389 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-22T03:45:18.389 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-22T03:45:18.389 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-22T03:45:18.389 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-22T03:45:18.389 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-22T03:45:18.389 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-22T03:45:18.390 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-22T03:45:18.390 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-22T03:45:18.390 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-22T03:45:18.390 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-22T03:45:18.390 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-22T03:45:18.390 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-22T03:45:18.390 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-22T03:45:18.390 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-22T03:45:18.390 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-22T03:45:18.390 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-22T03:45:18.390 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-22T03:45:18.390 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-22T03:45:18.390 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-22T03:45:18.390 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-22T03:45:18.391 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-22T03:45:18.392 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-22T03:45:18.392 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-22T03:45:18.392 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-22T03:45:18.392 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-22T03:45:18.392 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-22T03:45:18.392 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-22T03:45:18.392 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-22T03:45:18.392 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-22T03:45:18.490 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr 2024-09-22T03:45:19.081 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:25 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:45:19.172 INFO:teuthology.orchestra.run.smithi123.stdout:Package ceph-mgr-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:45:19.216 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:45:19.217 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-22T03:45:19.217 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:45:19.279 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr-dashboard 2024-09-22T03:45:19.321 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-22T03:45:19.321 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:19.321 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-22T03:45:19.321 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:45:19.321 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:45:19.321 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:45:19.321 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:45:19.321 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:45:19.321 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:45:19.322 INFO:teuthology.orchestra.run.smithi003.stdout: lua-5.4.4-4.el9.x86_64 2024-09-22T03:45:19.322 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-22T03:45:19.322 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-22T03:45:19.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-22T03:45:19.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-22T03:45:19.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-22T03:45:19.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-22T03:45:19.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-22T03:45:19.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-22T03:45:19.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-22T03:45:19.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-22T03:45:19.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-22T03:45:19.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-22T03:45:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-22T03:45:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-22T03:45:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-22T03:45:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-22T03:45:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-22T03:45:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-22T03:45:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply-3.11-14.el9.noarch 2024-09-22T03:45:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-22T03:45:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-22T03:45:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-22T03:45:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-22T03:45:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-22T03:45:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-22T03:45:19.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-22T03:45:19.324 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-22T03:45:19.324 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-22T03:45:19.324 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-22T03:45:19.324 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-22T03:45:19.324 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:19.324 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:45:19.585 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-base 2024-09-22T03:45:19.860 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:25 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:45:19.994 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:45:19.995 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:19.995 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-22T03:45:19.995 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:19.995 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-22T03:45:19.995 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 14 M 2024-09-22T03:45:19.995 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-22T03:45:19.995 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-22T03:45:19.996 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 34 k 2024-09-22T03:45:19.996 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 16 k 2024-09-22T03:45:19.996 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-22T03:45:19.996 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-22T03:45:19.996 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-22T03:45:19.996 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-22T03:45:19.996 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-22T03:45:19.996 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-22T03:45:19.996 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-22T03:45:19.996 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:19.996 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-22T03:45:19.996 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:19.996 INFO:teuthology.orchestra.run.smithi123.stdout:Install 11 Packages 2024-09-22T03:45:19.997 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:19.997 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 18 M 2024-09-22T03:45:19.997 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 156 M 2024-09-22T03:45:19.997 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-22T03:45:20.201 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:01:23 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:45:20.294 INFO:teuthology.orchestra.run.smithi003.stdout:Package ceph-base-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:45:20.339 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:45:20.340 INFO:teuthology.orchestra.run.smithi003.stdout:Nothing to do. 2024-09-22T03:45:20.340 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:45:20.409 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install cephadm 2024-09-22T03:45:20.555 INFO:teuthology.orchestra.run.smithi123.stdout:(1/11): ceph-grafana-dashboards-19.3.0-5092.g62 232 kB/s | 34 kB 00:00 2024-09-22T03:45:20.581 INFO:teuthology.orchestra.run.smithi123.stdout:(2/11): ceph-prometheus-alerts-19.3.0-5092.g62d 95 kB/s | 16 kB 00:00 2024-09-22T03:45:20.773 INFO:teuthology.orchestra.run.smithi123.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.2 MB/s | 269 kB 00:00 2024-09-22T03:45:20.873 INFO:teuthology.orchestra.run.smithi123.stdout:(4/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 5.4 MB/s | 548 kB 00:00 2024-09-22T03:45:20.898 INFO:teuthology.orchestra.run.smithi123.stdout:(5/11): grpc-data-1.46.7-10.el9.noarch.rpm 798 kB/s | 19 kB 00:00 2024-09-22T03:45:21.008 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:01:24 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:45:21.040 INFO:teuthology.orchestra.run.smithi123.stdout:(6/11): python3-grpcio-1.46.7-10.el9.x86_64.rpm 14 MB/s | 2.0 MB 00:00 2024-09-22T03:45:21.074 INFO:teuthology.orchestra.run.smithi123.stdout:(7/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.2 MB/s | 144 kB 00:00 2024-09-22T03:45:21.099 INFO:teuthology.orchestra.run.smithi123.stdout:(8/11): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-22T03:45:21.124 INFO:teuthology.orchestra.run.smithi123.stdout:(9/11): python3-routes-2.5.1-5.el9.noarch.rpm 7.3 MB/s | 188 kB 00:00 2024-09-22T03:45:21.144 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:45:21.144 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:21.144 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-09-22T03:45:21.144 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:21.144 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-22T03:45:21.144 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 784 k 2024-09-22T03:45:21.144 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:21.145 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-22T03:45:21.145 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:21.145 INFO:teuthology.orchestra.run.smithi003.stdout:Install 1 Package 2024-09-22T03:45:21.145 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:21.145 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 784 k 2024-09-22T03:45:21.145 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 791 k 2024-09-22T03:45:21.145 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-22T03:45:21.405 INFO:teuthology.orchestra.run.smithi003.stdout:cephadm-19.3.0-5092.g62da54b4.el9.noarch.rpm 3.0 MB/s | 784 kB 00:00 2024-09-22T03:45:21.406 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:45:21.406 INFO:teuthology.orchestra.run.smithi003.stdout:Total 2.9 MB/s | 784 kB 00:00 2024-09-22T03:45:21.406 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-22T03:45:21.412 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-22T03:45:21.412 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-22T03:45:21.418 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-22T03:45:21.418 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-22T03:45:21.496 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-22T03:45:21.541 INFO:teuthology.orchestra.run.smithi123.stdout:(10/11): ceph-mgr-dashboard-19.3.0-5092.g62da54 13 MB/s | 14 MB 00:01 2024-09-22T03:45:21.566 INFO:teuthology.orchestra.run.smithi123.stdout:(11/11): protobuf-compiler-3.14.0-13.el9.x86_64 876 kB/s | 863 kB 00:00 2024-09-22T03:45:21.570 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:45:21.570 INFO:teuthology.orchestra.run.smithi123.stdout:Total 12 MB/s | 18 MB 00:01 2024-09-22T03:45:21.601 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-22T03:45:21.620 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-22T03:45:21.620 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-22T03:45:21.887 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-22T03:45:21.887 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-22T03:45:21.989 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-22T03:45:22.078 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-22T03:45:22.455 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-22T03:45:22.463 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-22T03:45:22.547 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-09-22T03:45:22.578 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-09-22T03:45:22.686 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-09-22T03:45:22.736 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-22T03:45:22.736 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:22.736 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-22T03:45:22.736 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:45:22.736 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:22.736 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:45:22.812 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-09-22T03:45:22.864 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-09-22T03:45:22.932 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-immutable-object-cache 2024-09-22T03:45:22.949 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-09-22T03:45:22.991 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-09-22T03:45:23.028 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-09-22T03:45:23.061 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5092.g62da54b4.el9 9/11 2024-09-22T03:45:23.539 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:01:26 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:45:23.677 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:45:23.678 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:23.678 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repo Size 2024-09-22T03:45:23.678 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:23.678 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-22T03:45:23.678 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 158 k 2024-09-22T03:45:23.678 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:23.678 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-22T03:45:23.678 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:23.678 INFO:teuthology.orchestra.run.smithi003.stdout:Install 1 Package 2024-09-22T03:45:23.678 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:23.679 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 158 k 2024-09-22T03:45:23.679 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 644 k 2024-09-22T03:45:23.679 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-22T03:45:23.885 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-immutable-object-cache-19.3.0-5092.g62da54 764 kB/s | 158 kB 00:00 2024-09-22T03:45:23.886 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:45:23.886 INFO:teuthology.orchestra.run.smithi003.stdout:Total 759 kB/s | 158 kB 00:00 2024-09-22T03:45:23.886 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-22T03:45:23.897 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-22T03:45:23.897 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-22T03:45:23.958 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-22T03:45:23.959 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-22T03:45:24.122 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-22T03:45:24.211 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5092.g62da54b4. 1/1 2024-09-22T03:45:24.238 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5092.g62da54b4. 1/1 2024-09-22T03:45:24.238 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:45:24.238 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-22T03:45:24.238 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:25.229 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5092.g62da54b4. 1/1 2024-09-22T03:45:25.230 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:25.230 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-22T03:45:25.230 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:45:25.230 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:25.230 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:45:25.447 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-mgr 2024-09-22T03:45:26.050 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:01:29 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:45:26.143 INFO:teuthology.orchestra.run.smithi003.stdout:Package ceph-mgr-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:45:26.188 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:45:26.189 INFO:teuthology.orchestra.run.smithi003.stdout:Nothing to do. 2024-09-22T03:45:26.189 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:45:26.242 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5092.g62da54b4.el 10/11 2024-09-22T03:45:26.257 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-mgr-dashboard 2024-09-22T03:45:26.298 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5092.g62da54b4.el9.noa 11/11 2024-09-22T03:45:26.860 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:01:29 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:45:26.997 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:45:26.998 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:26.998 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-09-22T03:45:26.998 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:26.999 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-22T03:45:26.999 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 14 M 2024-09-22T03:45:26.999 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-09-22T03:45:26.999 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-22T03:45:26.999 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 34 k 2024-09-22T03:45:26.999 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 16 k 2024-09-22T03:45:27.000 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-22T03:45:27.000 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-22T03:45:27.000 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-22T03:45:27.000 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-22T03:45:27.000 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-22T03:45:27.000 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-22T03:45:27.001 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-22T03:45:27.001 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:27.001 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-22T03:45:27.001 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:27.001 INFO:teuthology.orchestra.run.smithi003.stdout:Install 11 Packages 2024-09-22T03:45:27.001 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:27.002 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 18 M 2024-09-22T03:45:27.003 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 156 M 2024-09-22T03:45:27.003 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-22T03:45:27.731 INFO:teuthology.orchestra.run.smithi003.stdout:(1/11): ceph-grafana-dashboards-19.3.0-5092.g62 119 kB/s | 34 kB 00:00 2024-09-22T03:45:27.756 INFO:teuthology.orchestra.run.smithi003.stdout:(2/11): ceph-prometheus-alerts-19.3.0-5092.g62d 53 kB/s | 16 kB 00:00 2024-09-22T03:45:27.973 INFO:teuthology.orchestra.run.smithi003.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.1 MB/s | 269 kB 00:00 2024-09-22T03:45:28.032 INFO:teuthology.orchestra.run.smithi003.stdout:(4/11): protobuf-compiler-3.14.0-13.el9.x86_64. 3.1 MB/s | 863 kB 00:00 2024-09-22T03:45:28.074 INFO:teuthology.orchestra.run.smithi003.stdout:(5/11): grpc-data-1.46.7-10.el9.noarch.rpm 468 kB/s | 19 kB 00:00 2024-09-22T03:45:28.107 INFO:teuthology.orchestra.run.smithi003.stdout:(6/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 4.0 MB/s | 548 kB 00:00 2024-09-22T03:45:28.132 INFO:teuthology.orchestra.run.smithi003.stdout:(7/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-22T03:45:28.158 INFO:teuthology.orchestra.run.smithi003.stdout:(8/11): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-22T03:45:28.191 INFO:teuthology.orchestra.run.smithi003.stdout:(9/11): python3-routes-2.5.1-5.el9.noarch.rpm 5.5 MB/s | 188 kB 00:00 2024-09-22T03:45:28.237 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5092.g62da54b4.el9.noa 11/11 2024-09-22T03:45:28.237 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5092.g62da54b4.el 1/11 2024-09-22T03:45:28.237 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5092.g62da54b4.el9.noa 2/11 2024-09-22T03:45:28.237 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5092.g62da54b4.el9 3/11 2024-09-22T03:45:28.237 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-09-22T03:45:28.237 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-09-22T03:45:28.237 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-09-22T03:45:28.237 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-09-22T03:45:28.238 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-09-22T03:45:28.238 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-09-22T03:45:28.238 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-09-22T03:45:28.300 INFO:teuthology.orchestra.run.smithi003.stdout:(10/11): python3-grpcio-1.46.7-10.el9.x86_64.rp 9.0 MB/s | 2.0 MB 00:00 2024-09-22T03:45:28.541 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-09-22T03:45:28.541 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:28.542 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-22T03:45:28.542 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-22T03:45:28.542 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:45:28.542 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:45:28.542 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:45:28.542 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-22T03:45:28.542 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-22T03:45:28.542 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-22T03:45:28.542 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-22T03:45:28.542 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-22T03:45:28.542 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-22T03:45:28.542 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-22T03:45:28.542 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:28.542 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:45:28.794 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-22T03:45:29.167 INFO:teuthology.orchestra.run.smithi003.stdout:(11/11): ceph-mgr-dashboard-19.3.0-5092.g62da54 8.2 MB/s | 14 MB 00:01 2024-09-22T03:45:29.172 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:45:29.172 INFO:teuthology.orchestra.run.smithi003.stdout:Total 8.4 MB/s | 18 MB 00:02 2024-09-22T03:45:29.206 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-22T03:45:29.228 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-22T03:45:29.228 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-22T03:45:29.369 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:35 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:45:29.500 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:45:29.501 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-09-22T03:45:29.501 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-22T03:45:29.501 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-09-22T03:45:29.501 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-22T03:45:29.501 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 7.4 M 2024-09-22T03:45:29.501 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-22T03:45:29.501 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-22T03:45:29.502 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-22T03:45:29.502 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-22T03:45:29.502 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-22T03:45:29.502 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-22T03:45:29.502 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-22T03:45:29.502 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-22T03:45:29.502 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-22T03:45:29.502 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-22T03:45:29.502 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-22T03:45:29.502 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-22T03:45:29.502 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:29.502 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-22T03:45:29.502 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-09-22T03:45:29.503 INFO:teuthology.orchestra.run.smithi123.stdout:Install 12 Packages 2024-09-22T03:45:29.503 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:29.503 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 38 M 2024-09-22T03:45:29.503 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 212 M 2024-09-22T03:45:29.503 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-22T03:45:29.506 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-22T03:45:29.507 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-22T03:45:30.071 INFO:teuthology.orchestra.run.smithi123.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 934 kB/s | 191 kB 00:00 2024-09-22T03:45:30.105 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-22T03:45:30.163 INFO:teuthology.orchestra.run.smithi123.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.6 MB/s | 801 kB 00:00 2024-09-22T03:45:30.188 INFO:teuthology.orchestra.run.smithi123.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-09-22T03:45:30.199 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-09-22T03:45:30.228 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-09-22T03:45:30.281 INFO:teuthology.orchestra.run.smithi123.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 199 kB/s | 18 kB 00:00 2024-09-22T03:45:30.334 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-09-22T03:45:30.373 INFO:teuthology.orchestra.run.smithi123.stdout:(5/12): ceph-mgr-diskprediction-local-19.3.0-50 15 MB/s | 7.4 MB 00:00 2024-09-22T03:45:30.406 INFO:teuthology.orchestra.run.smithi123.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 308 kB/s | 38 kB 00:00 2024-09-22T03:45:30.469 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-09-22T03:45:30.515 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-09-22T03:45:30.539 INFO:teuthology.orchestra.run.smithi123.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-22T03:45:30.603 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-09-22T03:45:30.660 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-09-22T03:45:30.695 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-09-22T03:45:30.728 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5092.g62da54b4.el9 9/11 2024-09-22T03:45:30.873 INFO:teuthology.orchestra.run.smithi123.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.2 MB/s | 3.0 MB 00:00 2024-09-22T03:45:30.956 INFO:teuthology.orchestra.run.smithi123.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.9 MB/s | 250 kB 00:00 2024-09-22T03:45:31.482 INFO:teuthology.orchestra.run.smithi123.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.5 MB/s | 4.9 MB 00:01 2024-09-22T03:45:31.716 INFO:teuthology.orchestra.run.smithi123.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.3 MB/s | 5.1 MB 00:01 2024-09-22T03:45:33.499 INFO:teuthology.orchestra.run.smithi123.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.4 MB/s | 16 MB 00:02 2024-09-22T03:45:33.500 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:45:33.500 INFO:teuthology.orchestra.run.smithi123.stdout:Total 9.6 MB/s | 38 MB 00:03 2024-09-22T03:45:33.860 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-22T03:45:33.887 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-22T03:45:33.887 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-22T03:45:34.124 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5092.g62da54b4.el 10/11 2024-09-22T03:45:34.181 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5092.g62da54b4.el9.noa 11/11 2024-09-22T03:45:34.310 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-22T03:45:34.310 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-22T03:45:35.074 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-22T03:45:35.141 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-22T03:45:35.188 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-22T03:45:35.288 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-22T03:45:35.331 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-22T03:45:35.515 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-22T03:45:35.547 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-22T03:45:35.623 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-22T03:45:35.996 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5092.g62da54b4.el9.noa 11/11 2024-09-22T03:45:35.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5092.g62da54b4.el 1/11 2024-09-22T03:45:35.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5092.g62da54b4.el9.noa 2/11 2024-09-22T03:45:35.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5092.g62da54b4.el9 3/11 2024-09-22T03:45:35.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-09-22T03:45:35.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-09-22T03:45:35.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-09-22T03:45:35.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-09-22T03:45:35.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-09-22T03:45:35.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-09-22T03:45:35.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-09-22T03:45:36.273 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-22T03:45:36.305 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-09-22T03:45:36.305 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:36.305 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-22T03:45:36.305 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-22T03:45:36.305 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:45:36.305 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:45:36.305 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:45:36.306 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-22T03:45:36.306 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-22T03:45:36.306 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-22T03:45:36.306 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-22T03:45:36.306 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-22T03:45:36.306 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-22T03:45:36.306 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-22T03:45:36.306 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:36.306 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:45:36.384 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-22T03:45:36.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-22T03:45:37.154 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:01:40 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:45:37.293 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:45:37.294 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-09-22T03:45:37.294 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-09-22T03:45:37.294 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-09-22T03:45:37.295 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-22T03:45:37.295 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 7.4 M 2024-09-22T03:45:37.295 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-09-22T03:45:37.295 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-22T03:45:37.295 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-22T03:45:37.295 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-22T03:45:37.295 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-22T03:45:37.295 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-22T03:45:37.295 INFO:teuthology.orchestra.run.smithi003.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-22T03:45:37.295 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-22T03:45:37.295 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-22T03:45:37.295 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-22T03:45:37.295 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-22T03:45:37.295 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-22T03:45:37.296 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:37.296 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-22T03:45:37.296 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-09-22T03:45:37.296 INFO:teuthology.orchestra.run.smithi003.stdout:Install 12 Packages 2024-09-22T03:45:37.296 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:37.296 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 38 M 2024-09-22T03:45:37.296 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 212 M 2024-09-22T03:45:37.296 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-22T03:45:37.574 INFO:teuthology.orchestra.run.smithi003.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 921 kB/s | 191 kB 00:00 2024-09-22T03:45:37.623 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-22T03:45:37.646 INFO:teuthology.orchestra.run.smithi003.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 453 kB/s | 33 kB 00:00 2024-09-22T03:45:37.705 INFO:teuthology.orchestra.run.smithi003.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-22T03:45:37.797 INFO:teuthology.orchestra.run.smithi003.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 199 kB/s | 18 kB 00:00 2024-09-22T03:45:37.847 INFO:teuthology.orchestra.run.smithi003.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 15 MB/s | 3.0 MB 00:00 2024-09-22T03:45:37.872 INFO:teuthology.orchestra.run.smithi003.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 511 kB/s | 38 kB 00:00 2024-09-22T03:45:37.990 INFO:teuthology.orchestra.run.smithi003.stdout:(7/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 35 MB/s | 4.9 MB 00:00 2024-09-22T03:45:38.015 INFO:teuthology.orchestra.run.smithi003.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-22T03:45:38.123 INFO:teuthology.orchestra.run.smithi003.stdout:(9/12): ceph-mgr-diskprediction-local-19.3.0-50 9.8 MB/s | 7.4 MB 00:00 2024-09-22T03:45:38.149 INFO:teuthology.orchestra.run.smithi003.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.8 MB/s | 250 kB 00:00 2024-09-22T03:45:38.216 INFO:teuthology.orchestra.run.smithi003.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-09-22T03:45:38.616 INFO:teuthology.orchestra.run.smithi003.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-22T03:45:38.617 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:45:38.617 INFO:teuthology.orchestra.run.smithi003.stdout:Total 29 MB/s | 38 MB 00:01 2024-09-22T03:45:38.649 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-22T03:45:38.678 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5092.g62da5 12/12 2024-09-22T03:45:38.983 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-22T03:45:39.011 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-22T03:45:39.011 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-22T03:45:39.467 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-22T03:45:39.467 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-22T03:45:40.265 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-22T03:45:40.327 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-22T03:45:40.365 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-22T03:45:40.471 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-22T03:45:40.515 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-22T03:45:40.712 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-22T03:45:40.747 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-22T03:45:40.827 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-22T03:45:41.153 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5092.g62da5 12/12 2024-09-22T03:45:41.153 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5092.g62da5 1/12 2024-09-22T03:45:41.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-22T03:45:41.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-22T03:45:41.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-22T03:45:41.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-22T03:45:41.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-22T03:45:41.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-22T03:45:41.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-22T03:45:41.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-22T03:45:41.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-22T03:45:41.154 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-22T03:45:41.522 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-22T03:45:41.568 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-22T03:45:41.568 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:41.568 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-22T03:45:41.568 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:45:41.568 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-22T03:45:41.568 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-22T03:45:41.568 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-22T03:45:41.568 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-22T03:45:41.568 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-22T03:45:41.568 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-22T03:45:41.568 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-22T03:45:41.568 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-22T03:45:41.568 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-22T03:45:41.569 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-22T03:45:41.569 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-22T03:45:41.569 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:41.569 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:45:41.641 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-22T03:45:41.977 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr-rook 2024-09-22T03:45:42.573 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:48 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:45:42.706 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:45:42.708 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-22T03:45:42.708 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:42.708 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-22T03:45:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 49 k 2024-09-22T03:45:42.708 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-22T03:45:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-22T03:45:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-22T03:45:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-22T03:45:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-22T03:45:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-22T03:45:42.708 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-22T03:45:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-22T03:45:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-22T03:45:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-22T03:45:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-22T03:45:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-22T03:45:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-22T03:45:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:42.709 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-22T03:45:42.709 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:42.709 INFO:teuthology.orchestra.run.smithi123.stdout:Install 13 Packages 2024-09-22T03:45:42.709 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:42.710 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 2.2 M 2024-09-22T03:45:42.710 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 26 M 2024-09-22T03:45:42.710 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-22T03:45:42.969 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-22T03:45:43.229 INFO:teuthology.orchestra.run.smithi123.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 360 kB/s | 26 kB 00:00 2024-09-22T03:45:43.254 INFO:teuthology.orchestra.run.smithi123.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 199 kB/s | 19 kB 00:00 2024-09-22T03:45:43.329 INFO:teuthology.orchestra.run.smithi123.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 3.2 MB/s | 222 kB 00:00 2024-09-22T03:45:43.354 INFO:teuthology.orchestra.run.smithi123.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-22T03:45:43.388 INFO:teuthology.orchestra.run.smithi123.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.7 MB/s | 279 kB 00:00 2024-09-22T03:45:43.413 INFO:teuthology.orchestra.run.smithi123.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 918 kB/s | 54 kB 00:00 2024-09-22T03:45:43.455 INFO:teuthology.orchestra.run.smithi123.stdout:(7/13): ceph-mgr-rook-19.3.0-5092.g62da54b4.el9 165 kB/s | 49 kB 00:00 2024-09-22T03:45:43.480 INFO:teuthology.orchestra.run.smithi123.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 350 kB/s | 32 kB 00:00 2024-09-22T03:45:43.513 INFO:teuthology.orchestra.run.smithi123.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 141 kB/s | 14 kB 00:00 2024-09-22T03:45:43.539 INFO:teuthology.orchestra.run.smithi123.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.5 MB/s | 59 kB 00:00 2024-09-22T03:45:43.572 INFO:teuthology.orchestra.run.smithi123.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 2.6 MB/s | 90 kB 00:00 2024-09-22T03:45:43.606 INFO:teuthology.orchestra.run.smithi123.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 8.6 MB/s | 1.0 MB 00:00 2024-09-22T03:45:43.632 INFO:teuthology.orchestra.run.smithi123.stdout:(13/13): python3-google-auth-2.34.0-1.el9.noarc 1.2 MB/s | 220 kB 00:00 2024-09-22T03:45:43.635 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:45:43.635 INFO:teuthology.orchestra.run.smithi123.stdout:Total 2.4 MB/s | 2.2 MB 00:00 2024-09-22T03:45:43.704 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-22T03:45:43.719 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-22T03:45:43.719 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-22T03:45:43.811 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-22T03:45:43.811 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-22T03:45:44.062 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-22T03:45:44.096 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5092.g62da5 12/12 2024-09-22T03:45:44.327 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-22T03:45:44.502 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-22T03:45:44.549 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-22T03:45:44.604 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-22T03:45:44.633 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-22T03:45:44.674 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-22T03:45:44.776 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-22T03:45:44.877 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-22T03:45:44.934 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-22T03:45:45.641 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-22T03:45:45.690 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-22T03:45:45.724 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-22T03:45:45.771 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-22T03:45:45.793 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.3.0-5092.g62da54b4.el9.noarch 13/13 2024-09-22T03:45:46.543 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5092.g62da54b4.el9.noarch 13/13 2024-09-22T03:45:46.543 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5092.g62da54b4.el9.noarch 1/13 2024-09-22T03:45:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-22T03:45:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-22T03:45:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-22T03:45:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-22T03:45:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-22T03:45:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-22T03:45:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-22T03:45:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-22T03:45:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-22T03:45:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-22T03:45:46.544 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-22T03:45:46.717 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5092.g62da5 12/12 2024-09-22T03:45:46.718 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5092.g62da5 1/12 2024-09-22T03:45:46.718 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-22T03:45:46.718 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-22T03:45:46.718 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-22T03:45:46.718 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-22T03:45:46.718 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-22T03:45:46.718 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-22T03:45:46.718 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-22T03:45:46.718 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-22T03:45:46.718 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-22T03:45:46.718 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-22T03:45:46.881 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-22T03:45:46.882 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:46.882 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-22T03:45:46.882 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:45:46.882 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-22T03:45:46.882 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-22T03:45:46.882 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-22T03:45:46.882 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-22T03:45:46.882 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-22T03:45:46.882 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-22T03:45:46.882 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-22T03:45:46.882 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-22T03:45:46.882 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-22T03:45:46.882 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-22T03:45:46.883 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-22T03:45:46.883 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-22T03:45:46.883 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:46.883 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:45:47.129 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-22T03:45:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:47.130 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-22T03:45:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:45:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-22T03:45:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-22T03:45:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-22T03:45:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-22T03:45:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-22T03:45:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-22T03:45:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-22T03:45:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-22T03:45:47.130 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-22T03:45:47.131 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-22T03:45:47.131 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-22T03:45:47.131 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:47.131 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:45:47.146 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr-cephadm 2024-09-22T03:45:47.520 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-mgr-rook 2024-09-22T03:45:47.726 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:53 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:45:47.859 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:45:47.859 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-22T03:45:47.860 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:47.860 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-22T03:45:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 154 k 2024-09-22T03:45:47.860 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-22T03:45:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-22T03:45:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-22T03:45:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-22T03:45:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-22T03:45:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-22T03:45:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:47.861 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-22T03:45:47.861 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:47.861 INFO:teuthology.orchestra.run.smithi123.stdout:Install 6 Packages 2024-09-22T03:45:47.861 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:47.861 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 7.0 M 2024-09-22T03:45:47.861 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 33 M 2024-09-22T03:45:47.861 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-22T03:45:48.127 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:01:51 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:45:48.233 INFO:teuthology.orchestra.run.smithi123.stdout:(1/6): ceph-mgr-cephadm-19.3.0-5092.g62da54b4.e 696 kB/s | 154 kB 00:00 2024-09-22T03:45:48.266 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:45:48.268 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:48.268 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-09-22T03:45:48.268 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:48.268 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-22T03:45:48.268 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 49 k 2024-09-22T03:45:48.268 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-09-22T03:45:48.268 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-22T03:45:48.268 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-22T03:45:48.268 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-22T03:45:48.268 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-22T03:45:48.268 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-22T03:45:48.268 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-22T03:45:48.269 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-22T03:45:48.269 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-22T03:45:48.269 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-22T03:45:48.269 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-22T03:45:48.269 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-22T03:45:48.269 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-22T03:45:48.269 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:48.269 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-22T03:45:48.269 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:48.269 INFO:teuthology.orchestra.run.smithi003.stdout:Install 13 Packages 2024-09-22T03:45:48.269 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:48.270 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 2.2 M 2024-09-22T03:45:48.270 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 26 M 2024-09-22T03:45:48.270 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-22T03:45:48.316 INFO:teuthology.orchestra.run.smithi123.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.4 MB/s | 548 kB 00:00 2024-09-22T03:45:48.342 INFO:teuthology.orchestra.run.smithi123.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-22T03:45:48.367 INFO:teuthology.orchestra.run.smithi123.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-22T03:45:48.553 INFO:teuthology.orchestra.run.smithi003.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 217 kB/s | 19 kB 00:00 2024-09-22T03:45:48.578 INFO:teuthology.orchestra.run.smithi003.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 226 kB/s | 26 kB 00:00 2024-09-22T03:45:48.642 INFO:teuthology.orchestra.run.smithi123.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 393 kB/s | 248 kB 00:00 2024-09-22T03:45:48.679 INFO:teuthology.orchestra.run.smithi003.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-09-22T03:45:48.704 INFO:teuthology.orchestra.run.smithi003.stdout:(4/13): ceph-mgr-rook-19.3.0-5092.g62da54b4.el9 206 kB/s | 49 kB 00:00 2024-09-22T03:45:48.729 INFO:teuthology.orchestra.run.smithi003.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.2 MB/s | 222 kB 00:00 2024-09-22T03:45:48.754 INFO:teuthology.orchestra.run.smithi003.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.6 MB/s | 279 kB 00:00 2024-09-22T03:45:48.813 INFO:teuthology.orchestra.run.smithi003.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 491 kB/s | 54 kB 00:00 2024-09-22T03:45:48.838 INFO:teuthology.orchestra.run.smithi003.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 295 kB/s | 32 kB 00:00 2024-09-22T03:45:48.864 INFO:teuthology.orchestra.run.smithi003.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 139 kB/s | 14 kB 00:00 2024-09-22T03:45:48.914 INFO:teuthology.orchestra.run.smithi003.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.1 MB/s | 220 kB 00:00 2024-09-22T03:45:48.939 INFO:teuthology.orchestra.run.smithi003.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 788 kB/s | 59 kB 00:00 2024-09-22T03:45:48.964 INFO:teuthology.orchestra.run.smithi003.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-22T03:45:49.006 INFO:teuthology.orchestra.run.smithi003.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.1 MB/s | 1.0 MB 00:00 2024-09-22T03:45:49.009 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:45:49.009 INFO:teuthology.orchestra.run.smithi003.stdout:Total 3.0 MB/s | 2.2 MB 00:00 2024-09-22T03:45:49.077 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-22T03:45:49.094 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-22T03:45:49.094 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-22T03:45:49.193 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-22T03:45:49.193 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-22T03:45:49.735 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-22T03:45:49.859 INFO:teuthology.orchestra.run.smithi123.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-09-22T03:45:49.861 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:45:49.861 INFO:teuthology.orchestra.run.smithi123.stdout:Total 3.5 MB/s | 7.0 MB 00:02 2024-09-22T03:45:49.915 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-22T03:45:49.932 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-22T03:45:49.943 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-22T03:45:49.944 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-22T03:45:49.976 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-22T03:45:50.040 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-22T03:45:50.045 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-22T03:45:50.047 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-22T03:45:50.076 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-22T03:45:50.118 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-22T03:45:50.224 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-22T03:45:50.335 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-22T03:45:50.362 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-22T03:45:50.386 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-22T03:45:50.533 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-22T03:45:50.578 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-22T03:45:51.105 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-22T03:45:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-22T03:45:51.197 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-22T03:45:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-22T03:45:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-22T03:45:51.264 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-22T03:45:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 6/6 2024-09-22T03:45:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-22T03:45:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-rook-2:19.3.0-5092.g62da54b4.el9.noarch 13/13 2024-09-22T03:45:52.053 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 6/6 2024-09-22T03:45:52.053 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 1/6 2024-09-22T03:45:52.053 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-22T03:45:52.053 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-22T03:45:52.053 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-22T03:45:52.053 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-22T03:45:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5092.g62da54b4.el9.noarch 13/13 2024-09-22T03:45:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5092.g62da54b4.el9.noarch 1/13 2024-09-22T03:45:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-22T03:45:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-22T03:45:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-22T03:45:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-22T03:45:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-22T03:45:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-22T03:45:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-22T03:45:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-22T03:45:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-22T03:45:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-22T03:45:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-22T03:45:52.338 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-22T03:45:52.338 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:52.338 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-22T03:45:52.338 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:45:52.338 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-22T03:45:52.338 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-22T03:45:52.338 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-22T03:45:52.339 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-22T03:45:52.339 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-22T03:45:52.339 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:52.339 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:45:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-22T03:45:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:52.519 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-22T03:45:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:45:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-22T03:45:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-22T03:45:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-22T03:45:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-22T03:45:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-22T03:45:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-22T03:45:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-22T03:45:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-22T03:45:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-22T03:45:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-22T03:45:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-22T03:45:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-22T03:45:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:52.520 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:45:52.589 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-fuse 2024-09-22T03:45:52.799 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-mgr-cephadm 2024-09-22T03:45:53.167 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:59 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:45:53.299 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:45:53.300 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:53.300 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-22T03:45:53.300 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:53.302 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-22T03:45:53.302 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 883 k 2024-09-22T03:45:53.302 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-22T03:45:53.302 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-22T03:45:53.302 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:53.302 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-22T03:45:53.302 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:53.302 INFO:teuthology.orchestra.run.smithi123.stdout:Install 2 Packages 2024-09-22T03:45:53.302 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:53.303 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 963 k 2024-09-22T03:45:53.303 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 4.1 M 2024-09-22T03:45:53.303 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-22T03:45:53.417 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:01:56 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:45:53.555 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:45:53.555 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:53.556 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-09-22T03:45:53.556 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:53.556 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-22T03:45:53.556 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 154 k 2024-09-22T03:45:53.556 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-09-22T03:45:53.556 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-22T03:45:53.556 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-22T03:45:53.556 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-22T03:45:53.556 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-22T03:45:53.556 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-22T03:45:53.556 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:53.556 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-22T03:45:53.556 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:53.557 INFO:teuthology.orchestra.run.smithi003.stdout:Install 6 Packages 2024-09-22T03:45:53.557 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:53.557 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 7.0 M 2024-09-22T03:45:53.557 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 33 M 2024-09-22T03:45:53.557 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-22T03:45:53.693 INFO:teuthology.orchestra.run.smithi123.stdout:(1/2): ceph-fuse-19.3.0-5092.g62da54b4.el9.x86_ 3.2 MB/s | 883 kB 00:00 2024-09-22T03:45:53.877 INFO:teuthology.orchestra.run.smithi123.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 175 kB/s | 80 kB 00:00 2024-09-22T03:45:53.877 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:45:53.878 INFO:teuthology.orchestra.run.smithi123.stdout:Total 1.6 MB/s | 963 kB 00:00 2024-09-22T03:45:53.889 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-22T03:45:53.913 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-22T03:45:53.913 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-22T03:45:53.993 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-22T03:45:53.994 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-22T03:45:54.184 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-22T03:45:54.283 INFO:teuthology.orchestra.run.smithi003.stdout:(1/6): ceph-mgr-cephadm-19.3.0-5092.g62da54b4.e 563 kB/s | 154 kB 00:00 2024-09-22T03:45:54.321 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-22T03:45:54.350 INFO:teuthology.orchestra.run.smithi003.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 8.1 MB/s | 548 kB 00:00 2024-09-22T03:45:54.375 INFO:teuthology.orchestra.run.smithi003.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-22T03:45:54.384 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 2/2 2024-09-22T03:45:54.650 INFO:teuthology.orchestra.run.smithi003.stdout:(4/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 387 kB/s | 248 kB 00:00 2024-09-22T03:45:54.900 INFO:teuthology.orchestra.run.smithi003.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 142 kB/s | 74 kB 00:00 2024-09-22T03:45:55.064 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 2/2 2024-09-22T03:45:55.064 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 1/2 2024-09-22T03:45:55.342 INFO:teuthology.orchestra.run.smithi003.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.5 MB/s | 6.0 MB 00:01 2024-09-22T03:45:55.345 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:45:55.345 INFO:teuthology.orchestra.run.smithi003.stdout:Total 3.9 MB/s | 7.0 MB 00:01 2024-09-22T03:45:55.347 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-22T03:45:55.347 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:55.347 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-22T03:45:55.348 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-22T03:45:55.348 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:55.348 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:45:55.420 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-22T03:45:55.433 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-22T03:45:55.433 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-22T03:45:55.541 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-22T03:45:55.541 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-22T03:45:55.642 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-volume 2024-09-22T03:45:55.859 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-22T03:45:56.042 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-22T03:45:56.095 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-22T03:45:56.227 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:02:02 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:45:56.366 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:45:56.367 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:56.367 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-22T03:45:56.367 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:56.367 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-22T03:45:56.367 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 285 k 2024-09-22T03:45:56.367 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:56.367 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-22T03:45:56.367 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:56.367 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-09-22T03:45:56.367 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:56.367 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 285 k 2024-09-22T03:45:56.367 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 1.3 M 2024-09-22T03:45:56.368 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-22T03:45:56.607 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-volume-19.3.0-5092.g62da54b4.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-22T03:45:56.607 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:45:56.607 INFO:teuthology.orchestra.run.smithi123.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-22T03:45:56.608 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-22T03:45:56.616 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-22T03:45:56.616 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-22T03:45:56.648 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-22T03:45:56.648 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-22T03:45:56.658 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-22T03:45:56.751 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-22T03:45:56.826 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-22T03:45:56.856 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 6/6 2024-09-22T03:45:56.865 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-22T03:45:56.933 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-22T03:45:56.949 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-22T03:45:56.949 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:45:56.949 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-22T03:45:56.949 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 6/6 2024-09-22T03:45:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 1/6 2024-09-22T03:45:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-22T03:45:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-22T03:45:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-22T03:45:57.674 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-22T03:45:57.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-22T03:45:57.934 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:57.935 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-22T03:45:57.935 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:45:57.935 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:57.935 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:45:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-22T03:45:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:57.950 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-22T03:45:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:45:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-22T03:45:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-22T03:45:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-22T03:45:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-22T03:45:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-22T03:45:57.951 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:57.951 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:45:58.215 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-fuse 2024-09-22T03:45:58.230 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install librados-devel 2024-09-22T03:45:58.816 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:02:01 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:45:58.823 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:02:04 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:45:58.955 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:45:58.955 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:58.955 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-09-22T03:45:58.956 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:58.956 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-22T03:45:58.956 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 883 k 2024-09-22T03:45:58.956 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-09-22T03:45:58.956 INFO:teuthology.orchestra.run.smithi003.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-22T03:45:58.956 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:58.956 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-22T03:45:58.956 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:45:58.956 INFO:teuthology.orchestra.run.smithi003.stdout:Install 2 Packages 2024-09-22T03:45:58.956 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:45:58.957 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 963 k 2024-09-22T03:45:58.957 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 4.1 M 2024-09-22T03:45:58.957 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-22T03:45:58.959 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:45:58.960 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:58.960 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repo Size 2024-09-22T03:45:58.960 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:58.960 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-22T03:45:58.960 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 125 k 2024-09-22T03:45:58.960 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:58.960 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-22T03:45:58.960 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:45:58.960 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-09-22T03:45:58.960 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:45:58.961 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 125 k 2024-09-22T03:45:58.961 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 445 k 2024-09-22T03:45:58.961 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-22T03:45:59.127 INFO:teuthology.orchestra.run.smithi003.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 656 kB/s | 80 kB 00:00 2024-09-22T03:45:59.193 INFO:teuthology.orchestra.run.smithi123.stdout:librados-devel-19.3.0-5092.g62da54b4.el9.x86_64 537 kB/s | 125 kB 00:00 2024-09-22T03:45:59.194 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:45:59.194 INFO:teuthology.orchestra.run.smithi123.stdout:Total 533 kB/s | 125 kB 00:00 2024-09-22T03:45:59.194 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-22T03:45:59.200 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-22T03:45:59.200 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-22T03:45:59.264 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-22T03:45:59.265 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-22T03:45:59.327 INFO:teuthology.orchestra.run.smithi003.stdout:(2/2): ceph-fuse-19.3.0-5092.g62da54b4.el9.x86_ 2.7 MB/s | 883 kB 00:00 2024-09-22T03:45:59.328 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:45:59.328 INFO:teuthology.orchestra.run.smithi003.stdout:Total 2.5 MB/s | 963 kB 00:00 2024-09-22T03:45:59.340 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-22T03:45:59.366 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-22T03:45:59.366 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-22T03:45:59.413 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-22T03:45:59.451 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-22T03:45:59.451 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-22T03:45:59.533 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:45:59.649 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-22T03:45:59.812 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-22T03:45:59.879 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 2/2 2024-09-22T03:46:00.018 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:00.304 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:00.304 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:00.304 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-22T03:46:00.304 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:46:00.304 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:00.304 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:46:00.536 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install libcephfs2 2024-09-22T03:46:00.623 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 2/2 2024-09-22T03:46:00.623 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 1/2 2024-09-22T03:46:00.932 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-22T03:46:00.932 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:00.932 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-22T03:46:00.932 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-22T03:46:00.932 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:00.932 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:46:01.127 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:02:07 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:46:01.218 INFO:teuthology.orchestra.run.smithi123.stdout:Package libcephfs2-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:46:01.225 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-volume 2024-09-22T03:46:01.264 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:46:01.265 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-22T03:46:01.265 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:46:01.324 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install libcephfs-devel 2024-09-22T03:46:01.845 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:02:04 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:46:01.902 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:02:07 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:46:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:46:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:01.985 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-09-22T03:46:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-22T03:46:01.985 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 285 k 2024-09-22T03:46:01.985 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-22T03:46:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:Install 1 Package 2024-09-22T03:46:01.986 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 285 k 2024-09-22T03:46:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 1.3 M 2024-09-22T03:46:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-22T03:46:02.034 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:46:02.035 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:46:02.035 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repo Size 2024-09-22T03:46:02.035 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:46:02.035 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-22T03:46:02.035 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 32 k 2024-09-22T03:46:02.035 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:02.035 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-22T03:46:02.035 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:46:02.035 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-09-22T03:46:02.035 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:02.036 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 32 k 2024-09-22T03:46:02.036 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 142 k 2024-09-22T03:46:02.036 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-22T03:46:02.279 INFO:teuthology.orchestra.run.smithi123.stdout:libcephfs-devel-19.3.0-5092.g62da54b4.el9.x86_6 130 kB/s | 32 kB 00:00 2024-09-22T03:46:02.280 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:46:02.280 INFO:teuthology.orchestra.run.smithi123.stdout:Total 129 kB/s | 32 kB 00:00 2024-09-22T03:46:02.280 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-22T03:46:02.284 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-22T03:46:02.284 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-22T03:46:02.308 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-22T03:46:02.308 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-22T03:46:02.374 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-volume-19.3.0-5092.g62da54b4.el9.noarch.rp 736 kB/s | 285 kB 00:00 2024-09-22T03:46:02.374 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:46:02.375 INFO:teuthology.orchestra.run.smithi003.stdout:Total 732 kB/s | 285 kB 00:00 2024-09-22T03:46:02.375 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-22T03:46:02.384 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-22T03:46:02.385 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-22T03:46:02.395 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-22T03:46:02.423 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-22T03:46:02.424 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-22T03:46:02.494 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:02.655 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-22T03:46:02.725 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-volume-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-22T03:46:02.742 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-volume-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-22T03:46:02.742 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:46:02.742 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-22T03:46:02.742 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:02.883 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:03.102 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:03.102 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:03.102 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-22T03:46:03.102 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:46:03.102 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:03.102 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:46:03.268 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install librados2 2024-09-22T03:46:03.785 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-volume-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-22T03:46:03.786 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:03.786 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-22T03:46:03.786 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-22T03:46:03.786 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:03.786 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:46:03.852 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:02:09 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:46:03.941 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:46:03.984 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:46:03.985 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-22T03:46:03.985 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:46:04.053 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install librbd1 2024-09-22T03:46:04.098 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install librados-devel 2024-09-22T03:46:04.625 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:02:10 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:46:04.709 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:02:07 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:46:04.713 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:46:04.758 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:46:04.759 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-22T03:46:04.759 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:46:04.819 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install python3-rados 2024-09-22T03:46:04.848 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:46:04.849 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:04.849 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repo Size 2024-09-22T03:46:04.849 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:04.849 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-22T03:46:04.849 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 125 k 2024-09-22T03:46:04.849 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:04.849 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-22T03:46:04.849 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:04.849 INFO:teuthology.orchestra.run.smithi003.stdout:Install 1 Package 2024-09-22T03:46:04.850 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:04.850 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 125 k 2024-09-22T03:46:04.850 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 445 k 2024-09-22T03:46:04.850 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-22T03:46:05.061 INFO:teuthology.orchestra.run.smithi003.stdout:librados-devel-19.3.0-5092.g62da54b4.el9.x86_64 589 kB/s | 125 kB 00:00 2024-09-22T03:46:05.062 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:46:05.062 INFO:teuthology.orchestra.run.smithi003.stdout:Total 585 kB/s | 125 kB 00:00 2024-09-22T03:46:05.062 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-22T03:46:05.069 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-22T03:46:05.069 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-22T03:46:05.136 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-22T03:46:05.137 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-22T03:46:05.296 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-22T03:46:05.395 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:02:11 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:46:05.404 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librados-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:05.481 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-rados-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:46:05.524 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:46:05.525 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-22T03:46:05.525 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:46:05.591 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install python3-rgw 2024-09-22T03:46:05.862 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:06.141 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:06.141 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:06.141 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-22T03:46:06.141 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:46:06.141 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:06.141 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:46:06.164 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:02:12 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:46:06.253 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-rgw-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:46:06.296 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:46:06.297 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-22T03:46:06.297 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:46:06.363 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install python3-cephfs 2024-09-22T03:46:06.380 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install libcephfs2 2024-09-22T03:46:06.940 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:02:12 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:46:06.991 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:02:09 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:46:07.030 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-cephfs-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:46:07.072 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:46:07.073 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-22T03:46:07.073 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:46:07.084 INFO:teuthology.orchestra.run.smithi003.stdout:Package libcephfs2-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:46:07.130 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:46:07.131 INFO:teuthology.orchestra.run.smithi003.stdout:Nothing to do. 2024-09-22T03:46:07.131 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:46:07.140 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install python3-rbd 2024-09-22T03:46:07.218 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install libcephfs-devel 2024-09-22T03:46:07.716 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:02:13 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:46:07.806 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-rbd-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:46:07.825 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:02:10 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:46:07.848 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:46:07.849 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-22T03:46:07.849 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:46:07.915 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install rbd-fuse 2024-09-22T03:46:07.963 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:46:07.964 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:07.964 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repo Size 2024-09-22T03:46:07.964 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:07.964 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-22T03:46:07.964 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 32 k 2024-09-22T03:46:07.965 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:07.965 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-22T03:46:07.965 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:07.965 INFO:teuthology.orchestra.run.smithi003.stdout:Install 1 Package 2024-09-22T03:46:07.965 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:07.965 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 32 k 2024-09-22T03:46:07.966 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 142 k 2024-09-22T03:46:07.966 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-22T03:46:08.114 INFO:teuthology.orchestra.run.smithi003.stdout:libcephfs-devel-19.3.0-5092.g62da54b4.el9.x86_6 212 kB/s | 32 kB 00:00 2024-09-22T03:46:08.115 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:46:08.115 INFO:teuthology.orchestra.run.smithi003.stdout:Total 210 kB/s | 32 kB 00:00 2024-09-22T03:46:08.115 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-22T03:46:08.119 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-22T03:46:08.119 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-22T03:46:08.135 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-22T03:46:08.135 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-22T03:46:08.231 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-22T03:46:08.351 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:08.496 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:02:14 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:46:08.628 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:46:08.629 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:46:08.629 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-09-22T03:46:08.629 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:46:08.629 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-22T03:46:08.630 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 91 k 2024-09-22T03:46:08.630 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:08.630 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-22T03:46:08.630 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:46:08.630 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-09-22T03:46:08.630 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:08.630 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 91 k 2024-09-22T03:46:08.631 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 321 k 2024-09-22T03:46:08.631 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-22T03:46:08.732 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:08.824 INFO:teuthology.orchestra.run.smithi123.stdout:rbd-fuse-19.3.0-5092.g62da54b4.el9.x86_64.rpm 467 kB/s | 91 kB 00:00 2024-09-22T03:46:08.825 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:46:08.825 INFO:teuthology.orchestra.run.smithi123.stdout:Total 464 kB/s | 91 kB 00:00 2024-09-22T03:46:08.825 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-22T03:46:08.833 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-22T03:46:08.834 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-22T03:46:08.897 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-22T03:46:08.898 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-22T03:46:08.956 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:08.956 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:08.956 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-22T03:46:08.956 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:46:08.956 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:08.956 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:46:09.042 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-22T03:46:09.139 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:09.151 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install librados2 2024-09-22T03:46:09.536 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:09.758 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:02:12 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:46:09.796 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:09.796 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:09.797 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-22T03:46:09.797 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:46:09.797 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:09.797 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:46:09.850 INFO:teuthology.orchestra.run.smithi003.stdout:Package librados2-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:46:09.897 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:46:09.898 INFO:teuthology.orchestra.run.smithi003.stdout:Nothing to do. 2024-09-22T03:46:09.898 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:46:09.966 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install librbd1 2024-09-22T03:46:10.039 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install rbd-mirror 2024-09-22T03:46:10.574 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:02:13 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:46:10.654 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:02:16 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:46:10.666 INFO:teuthology.orchestra.run.smithi003.stdout:Package librbd1-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:46:10.712 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:46:10.713 INFO:teuthology.orchestra.run.smithi003.stdout:Nothing to do. 2024-09-22T03:46:10.713 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:46:10.783 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install python3-rados 2024-09-22T03:46:10.795 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:46:10.796 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:46:10.796 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repo Size 2024-09-22T03:46:10.796 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:46:10.796 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-22T03:46:10.796 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 3.5 M 2024-09-22T03:46:10.796 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:10.796 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-22T03:46:10.796 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:46:10.797 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-09-22T03:46:10.797 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:10.797 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 3.5 M 2024-09-22T03:46:10.797 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 30 M 2024-09-22T03:46:10.797 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-22T03:46:11.144 INFO:teuthology.orchestra.run.smithi123.stdout:rbd-mirror-19.3.0-5092.g62da54b4.el9.x86_64.rpm 10 MB/s | 3.5 MB 00:00 2024-09-22T03:46:11.144 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:46:11.145 INFO:teuthology.orchestra.run.smithi123.stdout:Total 10 MB/s | 3.5 MB 00:00 2024-09-22T03:46:11.145 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-22T03:46:11.156 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-22T03:46:11.156 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-22T03:46:11.239 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-22T03:46:11.239 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-22T03:46:11.394 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:02:14 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:46:11.485 INFO:teuthology.orchestra.run.smithi003.stdout:Package python3-rados-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:46:11.532 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:46:11.533 INFO:teuthology.orchestra.run.smithi003.stdout:Nothing to do. 2024-09-22T03:46:11.534 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:46:11.600 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install python3-rgw 2024-09-22T03:46:11.820 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-22T03:46:11.886 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:11.912 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:11.912 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:46:11.912 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-22T03:46:11.912 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-22T03:46:11.912 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-22T03:46:11.912 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:12.204 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:02:15 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:46:12.297 INFO:teuthology.orchestra.run.smithi003.stdout:Package python3-rgw-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:46:12.343 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:46:12.344 INFO:teuthology.orchestra.run.smithi003.stdout:Nothing to do. 2024-09-22T03:46:12.344 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:46:12.418 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install python3-cephfs 2024-09-22T03:46:13.021 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:02:16 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:46:13.113 INFO:teuthology.orchestra.run.smithi003.stdout:Package python3-cephfs-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:46:13.159 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:46:13.160 INFO:teuthology.orchestra.run.smithi003.stdout:Nothing to do. 2024-09-22T03:46:13.160 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:46:13.162 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:13.162 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:13.162 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-22T03:46:13.162 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:46:13.162 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:13.162 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:46:13.232 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install python3-rbd 2024-09-22T03:46:13.405 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install rbd-nbd 2024-09-22T03:46:13.840 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:02:16 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:46:13.933 INFO:teuthology.orchestra.run.smithi003.stdout:Package python3-rbd-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-22T03:46:13.979 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:46:13.980 INFO:teuthology.orchestra.run.smithi003.stdout:Nothing to do. 2024-09-22T03:46:13.980 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:46:13.994 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:02:19 ago on Sun 22 Sep 2024 03:43:54 AM UTC. 2024-09-22T03:46:14.049 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install rbd-fuse 2024-09-22T03:46:14.130 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-22T03:46:14.130 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:46:14.130 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-09-22T03:46:14.130 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:46:14.131 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-22T03:46:14.131 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 176 k 2024-09-22T03:46:14.131 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:14.131 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-22T03:46:14.131 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-22T03:46:14.131 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-09-22T03:46:14.131 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:14.132 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 176 k 2024-09-22T03:46:14.132 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 685 k 2024-09-22T03:46:14.132 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-22T03:46:14.364 INFO:teuthology.orchestra.run.smithi123.stdout:rbd-nbd-19.3.0-5092.g62da54b4.el9.x86_64.rpm 762 kB/s | 176 kB 00:00 2024-09-22T03:46:14.364 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:46:14.365 INFO:teuthology.orchestra.run.smithi123.stdout:Total 756 kB/s | 176 kB 00:00 2024-09-22T03:46:14.365 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-22T03:46:14.374 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-22T03:46:14.374 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-22T03:46:14.439 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-22T03:46:14.439 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-22T03:46:14.594 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-22T03:46:14.654 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:02:17 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:46:14.686 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:14.793 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:46:14.793 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:14.793 INFO:teuthology.orchestra.run.smithi003.stdout: Package Architecture Version Repository Size 2024-09-22T03:46:14.794 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:14.794 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-22T03:46:14.794 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 91 k 2024-09-22T03:46:14.794 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:14.794 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-22T03:46:14.794 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:14.794 INFO:teuthology.orchestra.run.smithi003.stdout:Install 1 Package 2024-09-22T03:46:14.794 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:14.794 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 91 k 2024-09-22T03:46:14.794 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 321 k 2024-09-22T03:46:14.795 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-22T03:46:14.992 INFO:teuthology.orchestra.run.smithi003.stdout:rbd-fuse-19.3.0-5092.g62da54b4.el9.x86_64.rpm 460 kB/s | 91 kB 00:00 2024-09-22T03:46:14.992 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:46:14.993 INFO:teuthology.orchestra.run.smithi003.stdout:Total 457 kB/s | 91 kB 00:00 2024-09-22T03:46:14.993 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-22T03:46:15.003 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-22T03:46:15.003 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-22T03:46:15.070 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-22T03:46:15.070 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-22T03:46:15.122 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-nbd-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:15.224 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-22T03:46:15.342 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:15.368 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:15.368 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:15.368 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-22T03:46:15.368 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:46:15.368 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:15.368 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-22T03:46:15.590 DEBUG:teuthology.parallel:result is None 2024-09-22T03:46:15.751 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:16.011 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:16.011 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:16.011 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-22T03:46:16.012 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:46:16.012 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:16.012 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:46:16.278 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install rbd-mirror 2024-09-22T03:46:16.884 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:02:19 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:46:17.023 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:46:17.024 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repo Size 2024-09-22T03:46:17.024 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:17.024 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-22T03:46:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 3.5 M 2024-09-22T03:46:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:17.024 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-22T03:46:17.024 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:17.024 INFO:teuthology.orchestra.run.smithi003.stdout:Install 1 Package 2024-09-22T03:46:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:17.025 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 3.5 M 2024-09-22T03:46:17.025 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 30 M 2024-09-22T03:46:17.025 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-22T03:46:17.370 INFO:teuthology.orchestra.run.smithi003.stdout:rbd-mirror-19.3.0-5092.g62da54b4.el9.x86_64.rpm 10 MB/s | 3.5 MB 00:00 2024-09-22T03:46:17.370 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:46:17.371 INFO:teuthology.orchestra.run.smithi003.stdout:Total 10 MB/s | 3.5 MB 00:00 2024-09-22T03:46:17.371 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-22T03:46:17.384 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-22T03:46:17.384 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-22T03:46:17.476 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-22T03:46:17.476 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-22T03:46:18.094 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-22T03:46:18.162 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-mirror-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:18.187 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:18.188 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-22T03:46:18.188 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-22T03:46:18.188 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-22T03:46:18.188 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-22T03:46:18.188 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:19.462 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-mirror-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:19.462 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:19.462 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-22T03:46:19.462 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:46:19.462 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:19.462 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:46:19.670 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install rbd-nbd 2024-09-22T03:46:20.284 INFO:teuthology.orchestra.run.smithi003.stdout:Last metadata expiration check: 0:02:23 ago on Sun 22 Sep 2024 03:43:57 AM UTC. 2024-09-22T03:46:20.423 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-22T03:46:20.424 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:20.424 INFO:teuthology.orchestra.run.smithi003.stdout: Package Architecture Version Repository Size 2024-09-22T03:46:20.424 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:20.424 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-22T03:46:20.424 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 176 k 2024-09-22T03:46:20.424 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:20.424 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-22T03:46:20.424 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-22T03:46:20.424 INFO:teuthology.orchestra.run.smithi003.stdout:Install 1 Package 2024-09-22T03:46:20.424 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:20.425 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 176 k 2024-09-22T03:46:20.425 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 685 k 2024-09-22T03:46:20.425 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-22T03:46:20.773 INFO:teuthology.orchestra.run.smithi003.stdout:rbd-nbd-19.3.0-5092.g62da54b4.el9.x86_64.rpm 507 kB/s | 176 kB 00:00 2024-09-22T03:46:20.773 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-22T03:46:20.774 INFO:teuthology.orchestra.run.smithi003.stdout:Total 505 kB/s | 176 kB 00:00 2024-09-22T03:46:20.774 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-22T03:46:20.785 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-22T03:46:20.785 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-22T03:46:20.855 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-22T03:46:20.855 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-22T03:46:21.035 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-22T03:46:21.149 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-nbd-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:21.608 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-nbd-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:21.866 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-nbd-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-22T03:46:21.866 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-22T03:46:21.866 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-22T03:46:21.866 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-22T03:46:22.144 DEBUG:teuthology.parallel:result is None 2024-09-22T03:46:22.145 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=62da54b4b83a96ca9976b7e1d0c8272a564a2208 2024-09-22T03:46:22.281 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-22T03:46:22.303 INFO:teuthology.orchestra.run.smithi003.stdout:19.3.0-5092.g62da54b4.el9 2024-09-22T03:46:22.304 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5092.g62da54b4.el9 2024-09-22T03:46:22.304 INFO:teuthology.task.install:The correct ceph version 19.3.0-5092.g62da54b4 is installed. 2024-09-22T03:46:22.306 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=62da54b4b83a96ca9976b7e1d0c8272a564a2208 2024-09-22T03:46:22.439 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-22T03:46:22.463 INFO:teuthology.orchestra.run.smithi123.stdout:19.3.0-5092.g62da54b4.el9 2024-09-22T03:46:22.464 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5092.g62da54b4.el9 2024-09-22T03:46:22.464 INFO:teuthology.task.install:The correct ceph version 19.3.0-5092.g62da54b4 is installed. 2024-09-22T03:46:22.466 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-22T03:46:22.466 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:46:22.466 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-22T03:46:22.493 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-22T03:46:22.493 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-22T03:46:22.532 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-22T03:46:22.533 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:46:22.533 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/daemon-helper 2024-09-22T03:46:22.559 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-22T03:46:22.624 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-22T03:46:22.624 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-09-22T03:46:22.651 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-22T03:46:22.718 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-22T03:46:22.718 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:46:22.718 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-22T03:46:22.744 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-22T03:46:22.809 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-22T03:46:22.809 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-22T03:46:22.837 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-22T03:46:22.903 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-22T03:46:22.904 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:46:22.904 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/stdin-killer 2024-09-22T03:46:22.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-22T03:46:22.995 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-22T03:46:22.995 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-09-22T03:46:23.020 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-22T03:46:23.086 INFO:teuthology.run_tasks:Running task ceph... 2024-09-22T03:46:23.160 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-22T03:46:23.160 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /var/log/ceph 2024-09-22T03:46:23.162 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /var/log/ceph 2024-09-22T03:46:23.187 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-22T03:46:23.187 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-22T03:46:23.228 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-22T03:46:23.253 INFO:tasks.ceph:Creating extra log directories... 2024-09-22T03:46:23.253 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-22T03:46:23.294 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-22T03:46:23.333 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-22T03:46:23.333 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore ompap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-22T03:46:23.333 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi/7914369', 'branch': 'main', 'description': 'crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore tasks/rados_api_tests}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7914369', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore ompap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208'}}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 101, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 2950, 'sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208', 'sleep_before_teardown': 0, 'subset': '0/1', 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_62da54b4b83a96ca9976b7e1d0c8272a564a2208/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208', 'targets': {'smithi003.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNHcqJqBGP0EcDBMWF44nqaSlIl/Jzt0sq7WqIPSoJ+A1Yf9sko32i9UA7DoeM5QJQc60THmcnGvOz5a3SYGIH8=', 'smithi123.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDXtdxge8lX322TMSHVbyu23N4c3Zp9vEfKmeXoriZlRHiOP5d0CH3Vtq5QMUO2rwD15oBhvQ0XU8upeNBnxXSM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore ompap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True}, 'client': {'debug ms': 1}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['rados/test.sh --crimson', 'rados/test_pool_quota.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-21_20:56:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-22T03:46:23.334 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-22T03:46:23.361 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-22T03:46:23.376 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/run/ceph 2024-09-22T03:46:23.416 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/run/ceph 2024-09-22T03:46:23.440 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:46:23.441 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2024-09-22T03:46:23.495 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-22T03:46:23.496 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 2024-09-22T03:46:23.552 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-22T03:46:23.552 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:46:23.552 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-22T03:46:23.552 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:46:23.552 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:46:23.552 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-22 03:46:18.427693934 +0000 2024-09-22T03:46:23.552 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-22 03:43:16.139331221 +0000 2024-09-22T03:46:23.553 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-22 03:43:16.139331221 +0000 2024-09-22T03:46:23.553 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-22 03:43:16.139331221 +0000 2024-09-22T03:46:23.553 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-22T03:46:23.616 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-22T03:46:23.616 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-22T03:46:23.616 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000195519 s, 2.6 MB/s 2024-09-22T03:46:23.617 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-22T03:46:23.674 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 2024-09-22T03:46:23.730 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-22T03:46:23.730 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:46:23.730 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-22T03:46:23.730 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:46:23.730 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:46:23.731 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-22 03:46:18.428693922 +0000 2024-09-22T03:46:23.731 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-22 03:43:16.469327396 +0000 2024-09-22T03:46:23.731 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-22 03:43:16.469327396 +0000 2024-09-22T03:46:23.731 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-22 03:43:16.469327396 +0000 2024-09-22T03:46:23.731 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-22T03:46:23.794 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-22T03:46:23.794 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-22T03:46:23.794 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.0001828 s, 2.8 MB/s 2024-09-22T03:46:23.795 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-22T03:46:23.853 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 2024-09-22T03:46:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-22T03:46:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:46:23.909 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-22T03:46:23.909 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:46:23.909 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:46:23.909 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-22 03:46:18.428693922 +0000 2024-09-22T03:46:23.909 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-22 03:43:16.786323723 +0000 2024-09-22T03:46:23.909 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-22 03:43:16.786323723 +0000 2024-09-22T03:46:23.909 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-22 03:43:16.786323723 +0000 2024-09-22T03:46:23.909 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-22T03:46:23.973 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-22T03:46:23.973 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-22T03:46:23.973 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000214388 s, 2.4 MB/s 2024-09-22T03:46:23.975 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-22T03:46:24.031 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 2024-09-22T03:46:24.088 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-22T03:46:24.088 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:46:24.088 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-22T03:46:24.088 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:46:24.088 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:46:24.089 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-22 03:46:18.429693911 +0000 2024-09-22T03:46:24.089 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-22 03:43:17.115319912 +0000 2024-09-22T03:46:24.089 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-22 03:43:17.115319912 +0000 2024-09-22T03:46:24.089 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-22 03:43:17.115319912 +0000 2024-09-22T03:46:24.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-22T03:46:24.151 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-22T03:46:24.151 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-22T03:46:24.152 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000206732 s, 2.5 MB/s 2024-09-22T03:46:24.153 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-22T03:46:24.209 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-09-22T03:46:24.210 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-22T03:46:24.210 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-09-22T03:46:24.227 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-22T03:46:24.227 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-09-22T03:46:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-22T03:46:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:46:24.284 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-22T03:46:24.284 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:46:24.284 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:46:24.284 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-22 03:46:12.164279982 +0000 2024-09-22T03:46:24.284 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-22 03:43:08.602866827 +0000 2024-09-22T03:46:24.284 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-22 03:43:08.602866827 +0000 2024-09-22T03:46:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-22 03:43:08.602866827 +0000 2024-09-22T03:46:24.284 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-22T03:46:24.349 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-22T03:46:24.350 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-22T03:46:24.350 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000232258 s, 2.2 MB/s 2024-09-22T03:46:24.351 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-22T03:46:24.408 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-09-22T03:46:24.464 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-22T03:46:24.464 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:46:24.464 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-22T03:46:24.464 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:46:24.464 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:46:24.464 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-22 03:46:12.165279958 +0000 2024-09-22T03:46:24.464 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-22 03:43:08.935858791 +0000 2024-09-22T03:46:24.464 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-22 03:43:08.935858791 +0000 2024-09-22T03:46:24.464 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-22 03:43:08.935858791 +0000 2024-09-22T03:46:24.465 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-22T03:46:24.528 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-22T03:46:24.528 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-22T03:46:24.528 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000215753 s, 2.4 MB/s 2024-09-22T03:46:24.529 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-22T03:46:24.587 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-09-22T03:46:24.643 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-22T03:46:24.643 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:46:24.643 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-22T03:46:24.643 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:46:24.643 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:46:24.643 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-22 03:46:12.165279958 +0000 2024-09-22T03:46:24.643 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-22 03:43:09.274850610 +0000 2024-09-22T03:46:24.643 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-22 03:43:09.274850610 +0000 2024-09-22T03:46:24.643 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-22 03:43:09.274850610 +0000 2024-09-22T03:46:24.643 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-22T03:46:24.706 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-22T03:46:24.706 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-22T03:46:24.706 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000183468 s, 2.8 MB/s 2024-09-22T03:46:24.707 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-22T03:46:24.762 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-09-22T03:46:24.818 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-22T03:46:24.818 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T03:46:24.818 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-22T03:46:24.818 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T03:46:24.818 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-22T03:46:24.818 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-22 03:46:12.165279958 +0000 2024-09-22T03:46:24.818 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-22 03:43:09.605842622 +0000 2024-09-22T03:46:24.818 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-22 03:43:09.605842622 +0000 2024-09-22T03:46:24.819 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-22 03:43:09.605842622 +0000 2024-09-22T03:46:24.819 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-22T03:46:24.880 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-22T03:46:24.881 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-22T03:46:24.881 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000185868 s, 2.8 MB/s 2024-09-22T03:46:24.882 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-22T03:46:24.938 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-09-22T03:46:24.938 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-09-22T03:46:24.938 INFO:tasks.ceph:Generating config... 2024-09-22T03:46:24.939 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-09-22T03:46:24.939 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-09-22T03:46:24.939 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-09-22T03:46:24.940 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-09-22T03:46:24.940 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-09-22T03:46:24.940 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-22T03:46:24.940 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-22T03:46:24.940 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-22T03:46:24.940 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-22T03:46:24.940 INFO:tasks.ceph:[mon] osd pool default crimson = True 2024-09-22T03:46:24.940 INFO:tasks.ceph:[osd] debug monc = 20 2024-09-22T03:46:24.940 INFO:tasks.ceph:[osd] crimson alien thread cpu cores = 6-7 2024-09-22T03:46:24.940 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2024-09-22T03:46:24.940 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-22T03:46:24.940 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-22T03:46:24.940 INFO:tasks.ceph:[osd] debug seastore = 20 2024-09-22T03:46:24.940 INFO:tasks.ceph:[osd] debug seastore backref = 20 2024-09-22T03:46:24.941 INFO:tasks.ceph:[osd] debug seastore cache = 20 2024-09-22T03:46:24.941 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2024-09-22T03:46:24.941 INFO:tasks.ceph:[osd] debug seastore device = 20 2024-09-22T03:46:24.941 INFO:tasks.ceph:[osd] debug seastore epm = 20 2024-09-22T03:46:24.941 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2024-09-22T03:46:24.941 INFO:tasks.ceph:[osd] debug seastore journal = 20 2024-09-22T03:46:24.941 INFO:tasks.ceph:[osd] debug seastore lba = 20 2024-09-22T03:46:24.941 INFO:tasks.ceph:[osd] debug seastore odata = 20 2024-09-22T03:46:24.941 INFO:tasks.ceph:[osd] debug seastore ompap = 20 2024-09-22T03:46:24.941 INFO:tasks.ceph:[osd] debug seastore onode = 20 2024-09-22T03:46:24.941 INFO:tasks.ceph:[osd] debug seastore t = 20 2024-09-22T03:46:24.941 INFO:tasks.ceph:[osd] debug seastore tm = 20 2024-09-22T03:46:24.941 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-22T03:46:24.941 INFO:tasks.ceph:[osd] osd class default list = * 2024-09-22T03:46:24.942 INFO:tasks.ceph:[osd] osd class load list = * 2024-09-22T03:46:24.942 INFO:tasks.ceph:[osd] osd objectstore = seastore 2024-09-22T03:46:24.942 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-22T03:46:24.942 INFO:tasks.ceph:[client] debug ms = 1 2024-09-22T03:46:24.942 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2024-09-22T03:46:24.942 INFO:tasks.ceph:[global] ms client mode = crc 2024-09-22T03:46:24.942 INFO:tasks.ceph:[global] ms cluster mode = crc 2024-09-22T03:46:24.942 INFO:tasks.ceph:[global] ms mon client mode = crc 2024-09-22T03:46:24.942 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2024-09-22T03:46:24.942 INFO:tasks.ceph:[global] ms mon service mode = crc 2024-09-22T03:46:24.942 INFO:tasks.ceph:[global] ms service mode = crc 2024-09-22T03:46:24.942 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-22T03:46:24.942 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-22T03:46:24.942 INFO:tasks.ceph:[osd.0] crimson seastar cpu cores = 0-2 2024-09-22T03:46:24.943 INFO:tasks.ceph:[osd.1] crimson seastar cpu cores = 3-5 2024-09-22T03:46:24.943 INFO:tasks.ceph:[osd.2] crimson seastar cpu cores = 0-2 2024-09-22T03:46:24.943 INFO:tasks.ceph:[osd.3] crimson seastar cpu cores = 3-5 2024-09-22T03:46:24.943 INFO:tasks.ceph:Setting up mon.a... 2024-09-22T03:46:24.943 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-22T03:46:25.006 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.keyring 2024-09-22T03:46:25.010 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-22T03:46:25.066 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-22T03:46:25.092 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.3')] 2024-09-22T03:46:25.092 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.3', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore ompap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd blocked scrub grace period': 3600, 'osd class default list': '*', 'osd class load list': '*', 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'mon warn on pool no app': False, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1}, 'mon.a': {}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}} 2024-09-22T03:46:25.093 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:46:25.093 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-22T03:46:25.148 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.3 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:46:25.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:25.242+0000 7f07ba285f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:25.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:25.242+0000 7f07ba285f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:25.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:25.242+0000 7f07ba285f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:25.244 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:46:25.244 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: generated fsid 012c124d-1263-4b17-8b29-e14858734658 2024-09-22T03:46:25.244 INFO:teuthology.orchestra.run.smithi003.stdout:setting min_mon_release = quincy 2024-09-22T03:46:25.244 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 0 2024-09-22T03:46:25.244 INFO:teuthology.orchestra.run.smithi003.stdout:fsid 012c124d-1263-4b17-8b29-e14858734658 2024-09-22T03:46:25.244 INFO:teuthology.orchestra.run.smithi003.stdout:last_changed 2024-09-22T03:46:25.243483+0000 2024-09-22T03:46:25.244 INFO:teuthology.orchestra.run.smithi003.stdout:created 2024-09-22T03:46:25.243483+0000 2024-09-22T03:46:25.244 INFO:teuthology.orchestra.run.smithi003.stdout:min_mon_release 17 (quincy) 2024-09-22T03:46:25.244 INFO:teuthology.orchestra.run.smithi003.stdout:election_strategy: 1 2024-09-22T03:46:25.244 INFO:teuthology.orchestra.run.smithi003.stdout:0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-22T03:46:25.244 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-09-22T03:46:25.245 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-22T03:46:25.260 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 012c124d-1263-4b17-8b29-e14858734658... 2024-09-22T03:46:25.261 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-22T03:46:25.303 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-22T03:46:25.345 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-09-22T03:46:25.345 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2024-09-22T03:46:25.345 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-22T03:46:25.345 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2024-09-22T03:46:25.346 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.346 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2024-09-22T03:46:25.346 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.346 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2024-09-22T03:46:25.346 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.346 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2024-09-22T03:46:25.346 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2024-09-22T03:46:25.346 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.346 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2024-09-22T03:46:25.346 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2024-09-22T03:46:25.346 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.346 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-22T03:46:25.346 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2024-09-22T03:46:25.346 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.347 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2024-09-22T03:46:25.347 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-22T03:46:25.347 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.347 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2024-09-22T03:46:25.347 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.347 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2024-09-22T03:46:25.347 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2024-09-22T03:46:25.347 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2024-09-22T03:46:25.347 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2024-09-22T03:46:25.347 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2024-09-22T03:46:25.347 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2024-09-22T03:46:25.347 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2024-09-22T03:46:25.347 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-22T03:46:25.347 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-22T03:46:25.348 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2024-09-22T03:46:25.348 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.348 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-22T03:46:25.348 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.348 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2024-09-22T03:46:25.348 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.348 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2024-09-22T03:46:25.348 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.348 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2024-09-22T03:46:25.348 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2024-09-22T03:46:25.348 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2024-09-22T03:46:25.348 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = 172.21.15.3 2024-09-22T03:46:25.348 INFO:teuthology.orchestra.run.smithi003.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-09-22T03:46:25.348 INFO:teuthology.orchestra.run.smithi003.stdout: ms client mode = crc 2024-09-22T03:46:25.349 INFO:teuthology.orchestra.run.smithi003.stdout: ms cluster mode = crc 2024-09-22T03:46:25.349 INFO:teuthology.orchestra.run.smithi003.stdout: ms mon client mode = crc 2024-09-22T03:46:25.349 INFO:teuthology.orchestra.run.smithi003.stdout: ms mon cluster mode = crc 2024-09-22T03:46:25.349 INFO:teuthology.orchestra.run.smithi003.stdout: ms mon service mode = crc 2024-09-22T03:46:25.349 INFO:teuthology.orchestra.run.smithi003.stdout: ms service mode = crc 2024-09-22T03:46:25.349 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 012c124d-1263-4b17-8b29-e14858734658 2024-09-22T03:46:25.349 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.349 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2024-09-22T03:46:25.349 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2024-09-22T03:46:25.349 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.349 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2024-09-22T03:46:25.349 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2024-09-22T03:46:25.349 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2024-09-22T03:46:25.349 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.349 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2024-09-22T03:46:25.350 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2024-09-22T03:46:25.350 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.350 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2024-09-22T03:46:25.350 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2024-09-22T03:46:25.350 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2024-09-22T03:46:25.350 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug trim objects = true 2024-09-22T03:46:25.350 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.350 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2024-09-22T03:46:25.350 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2024-09-22T03:46:25.350 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.350 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2024-09-22T03:46:25.350 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.350 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2024-09-22T03:46:25.350 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.351 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2024-09-22T03:46:25.351 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.351 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2024-09-22T03:46:25.351 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2024-09-22T03:46:25.351 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.351 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2024-09-22T03:46:25.351 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2024-09-22T03:46:25.351 INFO:teuthology.orchestra.run.smithi003.stdout: debug monc = 20 2024-09-22T03:46:25.351 INFO:teuthology.orchestra.run.smithi003.stdout: crimson alien thread cpu cores = 6-7 2024-09-22T03:46:25.351 INFO:teuthology.orchestra.run.smithi003.stdout: crimson osd obc lru size = 10 2024-09-22T03:46:25.351 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-22T03:46:25.351 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2024-09-22T03:46:25.351 INFO:teuthology.orchestra.run.smithi003.stdout: debug seastore = 20 2024-09-22T03:46:25.351 INFO:teuthology.orchestra.run.smithi003.stdout: debug seastore backref = 20 2024-09-22T03:46:25.352 INFO:teuthology.orchestra.run.smithi003.stdout: debug seastore cache = 20 2024-09-22T03:46:25.352 INFO:teuthology.orchestra.run.smithi003.stdout: debug seastore cleaner = 20 2024-09-22T03:46:25.352 INFO:teuthology.orchestra.run.smithi003.stdout: debug seastore device = 20 2024-09-22T03:46:25.352 INFO:teuthology.orchestra.run.smithi003.stdout: debug seastore epm = 20 2024-09-22T03:46:25.352 INFO:teuthology.orchestra.run.smithi003.stdout: debug seastore fixedkv tree = 20 2024-09-22T03:46:25.352 INFO:teuthology.orchestra.run.smithi003.stdout: debug seastore journal = 20 2024-09-22T03:46:25.353 INFO:teuthology.orchestra.run.smithi003.stdout: debug seastore lba = 20 2024-09-22T03:46:25.353 INFO:teuthology.orchestra.run.smithi003.stdout: debug seastore odata = 20 2024-09-22T03:46:25.353 INFO:teuthology.orchestra.run.smithi003.stdout: debug seastore ompap = 20 2024-09-22T03:46:25.353 INFO:teuthology.orchestra.run.smithi003.stdout: debug seastore onode = 20 2024-09-22T03:46:25.353 INFO:teuthology.orchestra.run.smithi003.stdout: debug seastore t = 20 2024-09-22T03:46:25.353 INFO:teuthology.orchestra.run.smithi003.stdout: debug seastore tm = 20 2024-09-22T03:46:25.353 INFO:teuthology.orchestra.run.smithi003.stdout: osd blocked scrub grace period = 3600 2024-09-22T03:46:25.353 INFO:teuthology.orchestra.run.smithi003.stdout: osd class default list = * 2024-09-22T03:46:25.354 INFO:teuthology.orchestra.run.smithi003.stdout: osd class load list = * 2024-09-22T03:46:25.354 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = seastore 2024-09-22T03:46:25.354 INFO:teuthology.orchestra.run.smithi003.stdout: osd shutdown pgref assert = True 2024-09-22T03:46:25.354 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.354 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2024-09-22T03:46:25.354 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-22T03:46:25.354 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2024-09-22T03:46:25.354 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-22T03:46:25.354 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-22T03:46:25.355 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-22T03:46:25.355 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-22T03:46:25.355 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2024-09-22T03:46:25.355 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.355 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2024-09-22T03:46:25.355 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-22T03:46:25.355 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-22T03:46:25.355 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2024-09-22T03:46:25.355 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-22T03:46:25.356 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2024-09-22T03:46:25.356 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2024-09-22T03:46:25.356 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-22T03:46:25.356 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2024-09-22T03:46:25.356 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2024-09-22T03:46:25.356 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-22T03:46:25.356 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.356 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-22T03:46:25.356 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2024-09-22T03:46:25.357 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2024-09-22T03:46:25.357 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.357 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2024-09-22T03:46:25.357 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-22T03:46:25.357 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-22T03:46:25.357 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.357 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2024-09-22T03:46:25.358 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2024-09-22T03:46:25.358 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.358 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2024-09-22T03:46:25.358 INFO:teuthology.orchestra.run.smithi003.stdout: mon min osdmap epochs = 50 2024-09-22T03:46:25.358 INFO:teuthology.orchestra.run.smithi003.stdout: mon osdmap full prune interval = 2 2024-09-22T03:46:25.358 INFO:teuthology.orchestra.run.smithi003.stdout: mon osdmap full prune min = 15 2024-09-22T03:46:25.358 INFO:teuthology.orchestra.run.smithi003.stdout: mon osdmap full prune txsize = 2 2024-09-22T03:46:25.359 INFO:teuthology.orchestra.run.smithi003.stdout: paxos service trim min = 10 2024-09-22T03:46:25.359 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on pool no app = False 2024-09-22T03:46:25.359 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default crimson = True 2024-09-22T03:46:25.359 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:25.359 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2024-09-22T03:46:25.359 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2024-09-22T03:46:25.359 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2024-09-22T03:46:25.359 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2024-09-22T03:46:25.359 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-22T03:46:25.359 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-22T03:46:25.359 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-22T03:46:25.359 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2024-09-22T03:46:25.359 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.0] 2024-09-22T03:46:25.360 INFO:teuthology.orchestra.run.smithi003.stdout: crimson seastar cpu cores = 0-2 2024-09-22T03:46:25.360 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.1] 2024-09-22T03:46:25.360 INFO:teuthology.orchestra.run.smithi003.stdout: crimson seastar cpu cores = 3-5 2024-09-22T03:46:25.360 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.2] 2024-09-22T03:46:25.360 INFO:teuthology.orchestra.run.smithi003.stdout: crimson seastar cpu cores = 0-2 2024-09-22T03:46:25.360 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.3] 2024-09-22T03:46:25.360 INFO:teuthology.orchestra.run.smithi003.stdout: crimson seastar cpu cores = 3-5 2024-09-22T03:46:25.361 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2024-09-22T03:46:25.361 INFO:teuthology.orchestra.run.smithi123.stdout: chdir = "" 2024-09-22T03:46:25.361 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-22T03:46:25.361 INFO:teuthology.orchestra.run.smithi123.stdout: auth supported = cephx 2024-09-22T03:46:25.361 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.362 INFO:teuthology.orchestra.run.smithi123.stdout: filestore xattr use omap = true 2024-09-22T03:46:25.362 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.362 INFO:teuthology.orchestra.run.smithi123.stdout: mon clock drift allowed = 1.000 2024-09-22T03:46:25.362 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.362 INFO:teuthology.orchestra.run.smithi123.stdout: osd crush chooseleaf type = 0 2024-09-22T03:46:25.362 INFO:teuthology.orchestra.run.smithi123.stdout: auth debug = true 2024-09-22T03:46:25.362 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.362 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on old message = true 2024-09-22T03:46:25.362 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on bug = true 2024-09-22T03:46:25.362 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.362 INFO:teuthology.orchestra.run.smithi123.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-22T03:46:25.362 INFO:teuthology.orchestra.run.smithi123.stdout: mon pg warn max object skew = 0 2024-09-22T03:46:25.362 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.362 INFO:teuthology.orchestra.run.smithi123.stdout: # disable pg_autoscaler by default for new pools 2024-09-22T03:46:25.363 INFO:teuthology.orchestra.run.smithi123.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-22T03:46:25.363 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.363 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default size = 2 2024-09-22T03:46:25.363 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.363 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow primary affinity = true 2024-09-22T03:46:25.363 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow pg remap = true 2024-09-22T03:46:25.363 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on legacy crush tunables = false 2024-09-22T03:46:25.363 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on crush straw calc version zero = false 2024-09-22T03:46:25.363 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on no sortbitwise = false 2024-09-22T03:46:25.363 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on osd down out interval zero = false 2024-09-22T03:46:25.363 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on too few osds = false 2024-09-22T03:46:25.363 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-22T03:46:25.363 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-22T03:46:25.363 INFO:teuthology.orchestra.run.smithi123.stdout: mon_allow_pool_size_one = true 2024-09-22T03:46:25.363 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.364 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-22T03:46:25.364 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.364 INFO:teuthology.orchestra.run.smithi123.stdout: osd default data pool replay window = 5 2024-09-22T03:46:25.364 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.364 INFO:teuthology.orchestra.run.smithi123.stdout: mon allow pool delete = true 2024-09-22T03:46:25.364 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.364 INFO:teuthology.orchestra.run.smithi123.stdout: mon cluster log file level = debug 2024-09-22T03:46:25.364 INFO:teuthology.orchestra.run.smithi123.stdout: debug asserts on shutdown = true 2024-09-22T03:46:25.364 INFO:teuthology.orchestra.run.smithi123.stdout: mon health detail to clog = false 2024-09-22T03:46:25.364 INFO:teuthology.orchestra.run.smithi123.stdout: mon host = 172.21.15.3 2024-09-22T03:46:25.364 INFO:teuthology.orchestra.run.smithi123.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-09-22T03:46:25.364 INFO:teuthology.orchestra.run.smithi123.stdout: ms client mode = crc 2024-09-22T03:46:25.364 INFO:teuthology.orchestra.run.smithi123.stdout: ms cluster mode = crc 2024-09-22T03:46:25.364 INFO:teuthology.orchestra.run.smithi123.stdout: ms mon client mode = crc 2024-09-22T03:46:25.364 INFO:teuthology.orchestra.run.smithi123.stdout: ms mon cluster mode = crc 2024-09-22T03:46:25.364 INFO:teuthology.orchestra.run.smithi123.stdout: ms mon service mode = crc 2024-09-22T03:46:25.365 INFO:teuthology.orchestra.run.smithi123.stdout: ms service mode = crc 2024-09-22T03:46:25.365 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = 012c124d-1263-4b17-8b29-e14858734658 2024-09-22T03:46:25.365 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.365 INFO:teuthology.orchestra.run.smithi123.stdout:[osd] 2024-09-22T03:46:25.365 INFO:teuthology.orchestra.run.smithi123.stdout: osd journal size = 100 2024-09-22T03:46:25.365 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.365 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub load threshold = 5.0 2024-09-22T03:46:25.365 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub max interval = 600 2024-09-22T03:46:25.365 INFO:teuthology.orchestra.run.smithi123.stdout: osd mclock profile = high_recovery_ops 2024-09-22T03:46:25.365 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.365 INFO:teuthology.orchestra.run.smithi123.stdout: osd recover clone overlap = true 2024-09-22T03:46:25.365 INFO:teuthology.orchestra.run.smithi123.stdout: osd recovery max chunk = 1048576 2024-09-22T03:46:25.365 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.365 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug shutdown = true 2024-09-22T03:46:25.365 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug op order = true 2024-09-22T03:46:25.365 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify stray on activate = true 2024-09-22T03:46:25.366 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug trim objects = true 2024-09-22T03:46:25.366 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.366 INFO:teuthology.orchestra.run.smithi123.stdout: osd open classes on start = true 2024-09-22T03:46:25.366 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug pg log writeout = true 2024-09-22T03:46:25.366 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.366 INFO:teuthology.orchestra.run.smithi123.stdout: osd deep scrub update digest min age = 30 2024-09-22T03:46:25.366 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.366 INFO:teuthology.orchestra.run.smithi123.stdout: osd map max advance = 10 2024-09-22T03:46:25.366 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.366 INFO:teuthology.orchestra.run.smithi123.stdout: journal zero on create = true 2024-09-22T03:46:25.366 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.366 INFO:teuthology.orchestra.run.smithi123.stdout: filestore ondisk finisher threads = 3 2024-09-22T03:46:25.366 INFO:teuthology.orchestra.run.smithi123.stdout: filestore apply finisher threads = 3 2024-09-22T03:46:25.366 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.366 INFO:teuthology.orchestra.run.smithi123.stdout: bdev debug aio = true 2024-09-22T03:46:25.366 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug misdirected ops = true 2024-09-22T03:46:25.367 INFO:teuthology.orchestra.run.smithi123.stdout: debug monc = 20 2024-09-22T03:46:25.367 INFO:teuthology.orchestra.run.smithi123.stdout: crimson alien thread cpu cores = 6-7 2024-09-22T03:46:25.367 INFO:teuthology.orchestra.run.smithi123.stdout: crimson osd obc lru size = 10 2024-09-22T03:46:25.367 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-09-22T03:46:25.367 INFO:teuthology.orchestra.run.smithi123.stdout: debug osd = 20 2024-09-22T03:46:25.367 INFO:teuthology.orchestra.run.smithi123.stdout: debug seastore = 20 2024-09-22T03:46:25.367 INFO:teuthology.orchestra.run.smithi123.stdout: debug seastore backref = 20 2024-09-22T03:46:25.367 INFO:teuthology.orchestra.run.smithi123.stdout: debug seastore cache = 20 2024-09-22T03:46:25.367 INFO:teuthology.orchestra.run.smithi123.stdout: debug seastore cleaner = 20 2024-09-22T03:46:25.367 INFO:teuthology.orchestra.run.smithi123.stdout: debug seastore device = 20 2024-09-22T03:46:25.367 INFO:teuthology.orchestra.run.smithi123.stdout: debug seastore epm = 20 2024-09-22T03:46:25.367 INFO:teuthology.orchestra.run.smithi123.stdout: debug seastore fixedkv tree = 20 2024-09-22T03:46:25.367 INFO:teuthology.orchestra.run.smithi123.stdout: debug seastore journal = 20 2024-09-22T03:46:25.367 INFO:teuthology.orchestra.run.smithi123.stdout: debug seastore lba = 20 2024-09-22T03:46:25.368 INFO:teuthology.orchestra.run.smithi123.stdout: debug seastore odata = 20 2024-09-22T03:46:25.368 INFO:teuthology.orchestra.run.smithi123.stdout: debug seastore ompap = 20 2024-09-22T03:46:25.368 INFO:teuthology.orchestra.run.smithi123.stdout: debug seastore onode = 20 2024-09-22T03:46:25.368 INFO:teuthology.orchestra.run.smithi123.stdout: debug seastore t = 20 2024-09-22T03:46:25.368 INFO:teuthology.orchestra.run.smithi123.stdout: debug seastore tm = 20 2024-09-22T03:46:25.368 INFO:teuthology.orchestra.run.smithi123.stdout: osd blocked scrub grace period = 3600 2024-09-22T03:46:25.368 INFO:teuthology.orchestra.run.smithi123.stdout: osd class default list = * 2024-09-22T03:46:25.368 INFO:teuthology.orchestra.run.smithi123.stdout: osd class load list = * 2024-09-22T03:46:25.368 INFO:teuthology.orchestra.run.smithi123.stdout: osd objectstore = seastore 2024-09-22T03:46:25.368 INFO:teuthology.orchestra.run.smithi123.stdout: osd shutdown pgref assert = True 2024-09-22T03:46:25.368 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.368 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr] 2024-09-22T03:46:25.368 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-09-22T03:46:25.368 INFO:teuthology.orchestra.run.smithi123.stdout: debug mgr = 20 2024-09-22T03:46:25.368 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-09-22T03:46:25.368 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-09-22T03:46:25.369 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-09-22T03:46:25.369 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-09-22T03:46:25.369 INFO:teuthology.orchestra.run.smithi123.stdout: mgr/telemetry/nag = false 2024-09-22T03:46:25.369 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.369 INFO:teuthology.orchestra.run.smithi123.stdout:[mon] 2024-09-22T03:46:25.369 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-09-22T03:46:25.369 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-09-22T03:46:25.369 INFO:teuthology.orchestra.run.smithi123.stdout: debug paxos = 20 2024-09-22T03:46:25.369 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-09-22T03:46:25.369 INFO:teuthology.orchestra.run.smithi123.stdout: mon data avail warn = 5 2024-09-22T03:46:25.369 INFO:teuthology.orchestra.run.smithi123.stdout: mon mgr mkfs grace = 240 2024-09-22T03:46:25.369 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-09-22T03:46:25.369 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd reporter subtree level = osd 2024-09-22T03:46:25.369 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd prime pg temp = true 2024-09-22T03:46:25.369 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-09-22T03:46:25.369 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.369 INFO:teuthology.orchestra.run.smithi123.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-22T03:46:25.370 INFO:teuthology.orchestra.run.smithi123.stdout: auth mon ticket ttl = 660 # 11m 2024-09-22T03:46:25.370 INFO:teuthology.orchestra.run.smithi123.stdout: auth service ticket ttl = 240 # 4m 2024-09-22T03:46:25.370 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.370 INFO:teuthology.orchestra.run.smithi123.stdout: # don't complain about insecure global_id in the test suite 2024-09-22T03:46:25.370 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-22T03:46:25.370 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-22T03:46:25.370 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.370 INFO:teuthology.orchestra.run.smithi123.stdout: # 1m isn't quite enough 2024-09-22T03:46:25.370 INFO:teuthology.orchestra.run.smithi123.stdout: mon_down_mkfs_grace = 2m 2024-09-22T03:46:25.370 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.370 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_filestore_osds = false 2024-09-22T03:46:25.370 INFO:teuthology.orchestra.run.smithi123.stdout: mon min osdmap epochs = 50 2024-09-22T03:46:25.370 INFO:teuthology.orchestra.run.smithi123.stdout: mon osdmap full prune interval = 2 2024-09-22T03:46:25.370 INFO:teuthology.orchestra.run.smithi123.stdout: mon osdmap full prune min = 15 2024-09-22T03:46:25.370 INFO:teuthology.orchestra.run.smithi123.stdout: mon osdmap full prune txsize = 2 2024-09-22T03:46:25.371 INFO:teuthology.orchestra.run.smithi123.stdout: paxos service trim min = 10 2024-09-22T03:46:25.371 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on pool no app = False 2024-09-22T03:46:25.371 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default crimson = True 2024-09-22T03:46:25.371 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-22T03:46:25.371 INFO:teuthology.orchestra.run.smithi123.stdout:[client] 2024-09-22T03:46:25.371 INFO:teuthology.orchestra.run.smithi123.stdout: rgw cache enabled = true 2024-09-22T03:46:25.371 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable ops log = true 2024-09-22T03:46:25.371 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable usage log = true 2024-09-22T03:46:25.371 INFO:teuthology.orchestra.run.smithi123.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-22T03:46:25.371 INFO:teuthology.orchestra.run.smithi123.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-22T03:46:25.371 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-09-22T03:46:25.371 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.a] 2024-09-22T03:46:25.371 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.0] 2024-09-22T03:46:25.371 INFO:teuthology.orchestra.run.smithi123.stdout: crimson seastar cpu cores = 0-2 2024-09-22T03:46:25.371 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.1] 2024-09-22T03:46:25.372 INFO:teuthology.orchestra.run.smithi123.stdout: crimson seastar cpu cores = 3-5 2024-09-22T03:46:25.372 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.2] 2024-09-22T03:46:25.372 INFO:teuthology.orchestra.run.smithi123.stdout: crimson seastar cpu cores = 0-2 2024-09-22T03:46:25.372 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.3] 2024-09-22T03:46:25.372 INFO:teuthology.orchestra.run.smithi123.stdout: crimson seastar cpu cores = 3-5 2024-09-22T03:46:25.372 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-22T03:46:25.372 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-22T03:46:25.467 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-22T03:46:25.467 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:46:25.468 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-22T03:46:25.483 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:46:25.483 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-22T03:46:25.538 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi003.front.sepia.ceph.com 2024-09-22T03:46:25.538 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:46:25.538 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-22T03:46:25.538 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-22T03:46:25.613 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:46:25.613 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:46:25.669 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi123.front.sepia.ceph.com 2024-09-22T03:46:25.669 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-22T03:46:25.669 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-22T03:46:25.669 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-22T03:46:25.704 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-22T03:46:25.704 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:46:25.759 INFO:tasks.ceph:Setting up mon nodes... 2024-09-22T03:46:25.759 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-22T03:46:25.759 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-22T03:46:25.862 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-22T03:46:25.865 INFO:tasks.ceph:Setting up mds nodes... 2024-09-22T03:46:25.865 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-22T03:46:25.865 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-22T03:46:25.917 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-22T03:46:25.931 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-22T03:46:25.980 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-22T03:46:25.993 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-22T03:46:25.993 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-09-22T03:46:25.994 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-22T03:46:26.019 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-09-22T03:46:26.019 INFO:tasks.ceph:role: osd.0 2024-09-22T03:46:26.019 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-22T03:46:26.019 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-22T03:46:26.091 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T03:46:26.092 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T03:46:26.092 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T03:46:26.092 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-22T03:46:26.092 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T03:46:26.092 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-09-22T03:46:26.092 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T03:46:26.092 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-22T03:46:26.092 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T03:46:26.092 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T03:46:26.584 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2024-09-22T03:46:26.586 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-09-22T03:46:26.586 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-22T03:46:26.726 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-22T03:46:26.763 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-22T03:46:26.830 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-09-22T03:46:26.830 INFO:tasks.ceph:role: osd.1 2024-09-22T03:46:26.830 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-22T03:46:26.830 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-22T03:46:26.904 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T03:46:26.905 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T03:46:26.905 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T03:46:26.905 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-22T03:46:26.905 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T03:46:26.905 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-09-22T03:46:26.905 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T03:46:26.905 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-22T03:46:26.905 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T03:46:26.905 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T03:46:27.351 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2024-09-22T03:46:27.354 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-09-22T03:46:27.354 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-22T03:46:27.444 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-22T03:46:27.475 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:46:27.789 INFO:teuthology.orchestra.run.smithi003.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:46:27.790 INFO:teuthology.orchestra.run.smithi003.stderr:INFO 2024-09-22 03:46:27,790 seastar - Reactor backend: linux-aio 2024-09-22T03:46:27.802 INFO:teuthology.orchestra.run.smithi003.stderr:==27782==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:46:27.932 INFO:teuthology.orchestra.run.smithi003.stderr:INFO 2024-09-22 03:46:27,931 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-22T03:46:27.932 INFO:teuthology.orchestra.run.smithi003.stderr:INFO 2024-09-22 03:46:27,931 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 0-2 2024-09-22T03:46:27.932 INFO:teuthology.orchestra.run.smithi003.stderr:DEBUG 2024-09-22 03:46:27,932 [shard 0:main] seastar - reactor::drain 2024-09-22T03:46:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:----------------------------------------------------- 2024-09-22T03:46:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:Suppressions used: 2024-09-22T03:46:28.477 INFO:teuthology.orchestra.run.smithi003.stderr: count bytes template 2024-09-22T03:46:28.477 INFO:teuthology.orchestra.run.smithi003.stderr: 1 8 MallocExtension::Register 2024-09-22T03:46:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:----------------------------------------------------- 2024-09-22T03:46:28.477 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-22T03:46:28.529 INFO:teuthology.orchestra.run.smithi003.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:46:28.533 INFO:teuthology.orchestra.run.smithi003.stderr:INFO 2024-09-22 03:46:28,533 seastar - Reactor backend: linux-aio 2024-09-22T03:46:28.543 INFO:teuthology.orchestra.run.smithi003.stderr:==27780==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:46:28.608 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x8b56c 0x10d76c 0xdd31a 0x4b14c 0x5fccddd 0x5fcd08f 0x5fcd365 0x601ba63 0x601bc69 0x5f9bdb6 0x5f9bf90 0x5f9c51c 0x5de40ab 0x5dfc2ed 0x713a942 0x714c115 0x714d5b2 0x714d637 0x714d739 0x714d831 0x714da3e 0x714dc24 0x714df73 0x714e317 0x714e547 0x714e7b1 0x714ea7e 0x714ec9c 0x714ed54 0x714ee1a 0x714eefb 0x5c46eec 0x5c46ff6 0x5c470b2 0x5c4716e 0x5c4722a 0x5c47377 0x5c47502 0x5c478c3 0x5c616c6 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:28.609 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:28.848 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x377971b 0x3b8c09a 0x3b8c15e 0xb9e0fae 0xba1c4c3 0xba1c84e 0xb9bcc48 0xb9bfa98 0xb9aaa36 0xb9aae1a 0x9add3c3 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x931138d 0x931232c 0x93123af 0x9312434 0x9312536 0x931262e 0x93126b4 0x931276d 0x9312923 0x9312a5b 0x9312ad6 0x9312b5b 0x9312cb6 0x9312dd0 0x9312e55 0x9312f33 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:28.848 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0xfd4af 0xba056c0 0xba1435e 0xba14490 0xba1460c 0xba14db8 0xb9bc06f 0xb9bfa98 0xb9aaa36 0xb9aae1a 0x9add3c3 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x931138d 0x931232c 0x93123af 0x9312434 0x9312536 0x931262e 0x93126b4 0x931276d 0x9312923 0x9312a5b 0x9312ad6 0x9312b5b 0x9312cb6 0x9312dd0 0x9312e55 0x9312f33 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:28.848 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:28.849 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 90 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x3779709 0x3b8beba 0x3b8c15e 0xb9d336b 0xb9d3620 0xb9bb9f1 0xb9bfa98 0xb9aaa36 0xb9aae1a 0x9a95997 0x9ae1492 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x931138d 0x931232c 0x93123af 0x9312434 0x9312536 0x931262e 0x93126b4 0x931276d 0x9312923 0x9312a5b 0x9312ad6 0x9312b5b 0x9312cb6 0x9312dd0 0x9312e55 0x9312f33 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 90 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x4197f 0x41c92 0x41f68 0x4214e 0x3ef7b 0xfd531 0xba0b99f 0xba0ba6f 0xb9bb43d 0xb9bfa98 0xb9aaa36 0xb9aae1a 0x9a95997 0x9ae1492 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x931138d 0x931232c 0x93123af 0x9312434 0x9312536 0x931262e 0x93126b4 0x931276d 0x9312923 0x9312a5b 0x9312ad6 0x9312b5b 0x9312cb6 0x9312dd0 0x9312e55 0x9312f33 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x4bf6a 0xb9be0f5 0xb9aaa36 0xb9aae1a 0x9add3c3 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x9258152 0x925866e 0x92587f7 0x9258a3e 0x9258d16 0x9258dcb 0x9576088 0x95762ed 0x95ae2d3 0x96b28be 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:28.964 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 123 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x10bf9f 0x10a294 0xfb55e 0x3767e9b 0x9826b93 0x9add2fa 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x9258152 0x925866e 0x92587f7 0x9258a3e 0x9258d16 0x9258dcb 0x9576088 0x95762ed 0x95ae2d3 0x96b28be 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:28.964 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:29.007 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x4218a 0x3ef7b 0xfd531 0xba0913f 0xb9ae311 0x9add337 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x9258152 0x925866e 0x92587f7 0x9258a3e 0x9258d16 0x9258dcb 0x9576088 0x95762ed 0x95ae2d3 0x96b28be 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:29.007 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffa639b810 0xffffffffa608c811 0xffffffffa6cd1db2 0xffffffffa6e00bb2 2024-09-22T03:46:29.007 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x8b56c 0x10d76c 0xdd31a 0x4ab1c 0x3ac7302 0x3b8c153 0xb9e0fae 0xba1c4c3 0xba1c84e 0xb9bcc48 0xb9bfa98 0xb9aaa36 0xb9aae1a 0x9add3c3 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x9258152 0x925866e 0x92587f7 0x9258a3e 0x9258d16 0x9258dcb 0x9576088 0x95762ed 0x95ae2d3 0x96b28be 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:29.007 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:29.064 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 123 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x4b1c5 0x3765ac4 0x3765bd5 0xb9c7f79 0xb9a9cf1 0xb9be5cb 0xb9aaa36 0xb9aae1a 0x9add3c3 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x9258152 0x925866e 0x92587f7 0x9258a3e 0x9258d16 0x9258dcb 0x9576088 0x95762ed 0x95ae2d3 0x96b28be 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:29.064 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:29.065 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 123 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0xdd327 0x4b14c 0x3b8bfdd 0x3b8c15e 0xb9cc607 0xb9cd8da 0xb9cdb54 0xb9cdcfc 0xba15051 0xb9bc06f 0xb9bfa98 0xb9aaa36 0xb9aae1a 0x9add3c3 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x9258152 0x925866e 0x92587f7 0x9258a3e 0x9258d16 0x9258dcb 0x9576088 0x95762ed 0x95ae2d3 0x96b28be 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:29.065 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:29.104 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x5525170 0x9e16cf9 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:29.104 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:29.161 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 122 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x552539f 0x9e16d0e 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:29.161 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:29.200 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x133c1f 0xdd332 0x4ab1c 0x5525eb7 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:29.200 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:29.203 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x133ccf 0xdd332 0x4ab1c 0x5525eb7 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:29.203 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:30.705 INFO:teuthology.orchestra.run.smithi003.stderr:----------------------------------------------------- 2024-09-22T03:46:30.705 INFO:teuthology.orchestra.run.smithi003.stderr:Suppressions used: 2024-09-22T03:46:30.706 INFO:teuthology.orchestra.run.smithi003.stderr: count bytes template 2024-09-22T03:46:30.706 INFO:teuthology.orchestra.run.smithi003.stderr: 1 8 MallocExtension::Register 2024-09-22T03:46:30.706 INFO:teuthology.orchestra.run.smithi003.stderr:----------------------------------------------------- 2024-09-22T03:46:30.706 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-22T03:46:30.734 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-22T03:46:30.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:46:31.058 INFO:teuthology.orchestra.run.smithi003.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:46:31.063 INFO:teuthology.orchestra.run.smithi003.stderr:INFO 2024-09-22 03:46:31,059 seastar - Reactor backend: linux-aio 2024-09-22T03:46:31.064 INFO:teuthology.orchestra.run.smithi003.stderr:==27830==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:46:31.172 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x3612fee 0x5fc01c3 0x5fc05c8 0x5fc09aa 0x601bbd4 0x5f9bdb6 0x5f9bf90 0x5f9c51c 0x5de40ab 0x5dfc2ed 0x713a942 0x714c115 0x714d5b2 0x714d637 0x714d739 0x714d831 0x714da3e 0x714dc24 0x714df73 0x714e317 0x714e547 0x714e7b1 0x714ea7e 0x714ec9c 0x714ed54 0x714ee1a 0x714eefb 0x5c46eec 0x5c46ff6 0x5c470b2 0x5c4716e 0x5c4722a 0x5c47377 0x5c47502 0x5c478c3 0x5c616c6 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3961ae8 0x39632c2 0x3803c18 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:31.172 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:31.194 INFO:teuthology.orchestra.run.smithi003.stderr:INFO 2024-09-22 03:46:31,193 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-22T03:46:31.194 INFO:teuthology.orchestra.run.smithi003.stderr:INFO 2024-09-22 03:46:31,193 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 3-5 2024-09-22T03:46:31.194 INFO:teuthology.orchestra.run.smithi003.stderr:DEBUG 2024-09-22 03:46:31,194 [shard 0:main] seastar - reactor::drain 2024-09-22T03:46:31.721 INFO:teuthology.orchestra.run.smithi003.stderr:----------------------------------------------------- 2024-09-22T03:46:31.721 INFO:teuthology.orchestra.run.smithi003.stderr:Suppressions used: 2024-09-22T03:46:31.721 INFO:teuthology.orchestra.run.smithi003.stderr: count bytes template 2024-09-22T03:46:31.721 INFO:teuthology.orchestra.run.smithi003.stderr: 1 8 MallocExtension::Register 2024-09-22T03:46:31.721 INFO:teuthology.orchestra.run.smithi003.stderr:----------------------------------------------------- 2024-09-22T03:46:31.722 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-22T03:46:31.777 INFO:teuthology.orchestra.run.smithi003.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:46:31.781 INFO:teuthology.orchestra.run.smithi003.stderr:INFO 2024-09-22 03:46:31,781 seastar - Reactor backend: linux-aio 2024-09-22T03:46:31.791 INFO:teuthology.orchestra.run.smithi003.stderr:==27828==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:46:31.856 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x4abc8 0x5fccc96 0x5fcce5e 0x5fcd08f 0x5fcd365 0x601ba63 0x601bc69 0x5f9bdb6 0x5f9bf90 0x5f9c51c 0x5de40ab 0x5dfc2ed 0x713a942 0x714c115 0x714d5b2 0x714d637 0x714d739 0x714d831 0x714da3e 0x714dc24 0x714df73 0x714e317 0x714e547 0x714e7b1 0x714ea7e 0x714ec9c 0x714ed54 0x714ee1a 0x714eefb 0x5c46eec 0x5c46ff6 0x5c470b2 0x5c4716e 0x5c4722a 0x5c47377 0x5c47502 0x5c478c3 0x5c616c6 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:31.856 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:32.094 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x133d18 0xdd332 0x4b14c 0x3b8bfdd 0x3b8c15e 0xb9cc43a 0xb9cd8da 0xb9cdb54 0xb9cdcfc 0xb9e53f4 0xba14771 0xba14db8 0xb9bc06f 0xb9bfa98 0xb9aaa36 0xb9aae1a 0x9add3c3 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x931138d 0x931232c 0x93123af 0x9312434 0x9312536 0x931262e 0x93126b4 0x931276d 0x9312923 0x9312a5b 0x9312ad6 0x9312b5b 0x9312cb6 0x9312dd0 0x9312e55 0x9312f33 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:32.094 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x41e59 0x4214e 0x3ef7b 0xfd531 0xb83682d 0xba16ccc 0xba16dfe 0xba16f7a 0xba17729 0xb9b1eb4 0x9826b31 0x9add2fa 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x931138d 0x931232c 0x93123af 0x9312434 0x9312536 0x931262e 0x93126b4 0x931276d 0x9312923 0x9312a5b 0x9312ad6 0x9312b5b 0x9312cb6 0x9312dd0 0x9312e55 0x9312f33 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:32.095 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:32.095 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:32.152 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 90 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x8b571 0x10d76c 0xdd31a 0x4b14c 0x3b8bfdd 0x3b8c15e 0xb9e1256 0xba1c4c3 0xba1c84e 0xb9bcc48 0xb9bfa98 0xb9aaa36 0xb9aae1a 0x9a95997 0x9ae1492 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x931138d 0x931232c 0x93123af 0x9312434 0x9312536 0x931262e 0x93126b4 0x931276d 0x9312923 0x9312a5b 0x9312ad6 0x9312b5b 0x9312cb6 0x9312dd0 0x9312e55 0x9312f33 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:32.152 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 90 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x36105be 0x36107c8 0xb837958 0xba09327 0xb9be52d 0xb9aaa36 0xb9aae1a 0x9a95997 0x9ae1626 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x931138d 0x931232c 0x93123af 0x9312434 0x9312536 0x931262e 0x93126b4 0x931276d 0x9312923 0x9312a5b 0x9312ad6 0x9312b5b 0x9312cb6 0x9312dd0 0x9312e55 0x9312f33 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:32.152 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:32.152 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:32.356 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x5525d72 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:32.356 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:32.414 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 122 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x9e16d06 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:32.414 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x4ab09 0x5525eb7 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:32.465 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:32.471 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x5525179 0x9e16cf9 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:32.472 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:32.521 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 230 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x4ab6e 0x5525eb7 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:32.522 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:32.522 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 122 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0xdd323 0x4ab1c 0x5525eb7 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:32.523 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:32.529 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 122 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x4ab37 0x5525eb7 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:32.529 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:32.630 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 230 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x156e8 0xdd312 0x4ab1c 0x5525eb7 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:32.630 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:32.636 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 230 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x5525e50 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:32.637 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:33.012 INFO:teuthology.orchestra.run.smithi003.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x10251f 0x10d76c 0xdd31a 0x4ab1c 0x5525eb7 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:33.012 INFO:teuthology.orchestra.run.smithi003.stderr:kernel callstack: 2024-09-22T03:46:33.976 INFO:teuthology.orchestra.run.smithi003.stderr:----------------------------------------------------- 2024-09-22T03:46:33.976 INFO:teuthology.orchestra.run.smithi003.stderr:Suppressions used: 2024-09-22T03:46:33.976 INFO:teuthology.orchestra.run.smithi003.stderr: count bytes template 2024-09-22T03:46:33.976 INFO:teuthology.orchestra.run.smithi003.stderr: 1 8 MallocExtension::Register 2024-09-22T03:46:33.976 INFO:teuthology.orchestra.run.smithi003.stderr:----------------------------------------------------- 2024-09-22T03:46:33.976 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-22T03:46:34.003 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-22T03:46:34.028 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-22T03:46:34.058 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-09-22T03:46:34.058 INFO:tasks.ceph:role: osd.2 2024-09-22T03:46:34.058 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-22T03:46:34.058 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-22T03:46:34.134 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T03:46:34.134 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T03:46:34.134 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T03:46:34.134 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-22T03:46:34.134 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T03:46:34.134 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-09-22T03:46:34.134 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T03:46:34.135 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-22T03:46:34.135 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T03:46:34.135 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T03:46:34.655 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-09-22T03:46:34.657 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-09-22T03:46:34.657 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-09-22T03:46:34.797 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-22T03:46:34.834 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-22T03:46:34.904 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-09-22T03:46:34.904 INFO:tasks.ceph:role: osd.3 2024-09-22T03:46:34.904 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-22T03:46:34.904 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-22T03:46:34.984 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T03:46:34.984 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T03:46:34.985 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T03:46:34.985 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-22T03:46:34.985 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T03:46:34.985 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-09-22T03:46:34.985 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T03:46:34.985 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-22T03:46:34.985 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T03:46:34.985 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T03:46:35.480 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-09-22T03:46:35.482 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-09-22T03:46:35.482 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2024-09-22T03:46:35.576 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-22T03:46:35.608 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:46:35.923 INFO:teuthology.orchestra.run.smithi123.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:46:35.924 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-22 03:46:35,923 seastar - Reactor backend: linux-aio 2024-09-22T03:46:35.939 INFO:teuthology.orchestra.run.smithi123.stderr:==27623==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:46:36.073 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-22 03:46:36,072 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-22T03:46:36.073 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-22 03:46:36,072 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 0-2 2024-09-22T03:46:36.073 INFO:teuthology.orchestra.run.smithi123.stderr:DEBUG 2024-09-22 03:46:36,073 [shard 0:main] seastar - reactor::drain 2024-09-22T03:46:36.603 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-22T03:46:36.603 INFO:teuthology.orchestra.run.smithi123.stderr:Suppressions used: 2024-09-22T03:46:36.603 INFO:teuthology.orchestra.run.smithi123.stderr: count bytes template 2024-09-22T03:46:36.603 INFO:teuthology.orchestra.run.smithi123.stderr: 1 8 MallocExtension::Register 2024-09-22T03:46:36.603 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-22T03:46:36.603 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-22T03:46:36.666 INFO:teuthology.orchestra.run.smithi123.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:46:36.669 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-22 03:46:36,669 seastar - Reactor backend: linux-aio 2024-09-22T03:46:36.679 INFO:teuthology.orchestra.run.smithi123.stderr:==27621==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:46:36.959 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x133c3f 0xdd332 0x4ab1c 0xb9cc4a3 0xb9cd8da 0xb9cdb54 0xb9cdcfc 0xb9cfab7 0xb9cff02 0xb9bbe2c 0xb9bfa98 0xb9aaa36 0xb9aae1a 0x9add3c3 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x931138d 0x931232c 0x93123af 0x9312434 0x9312536 0x931262e 0x93126b4 0x931276d 0x9312923 0x9312a5b 0x9312ad6 0x9312b5b 0x9312cb6 0x9312dd0 0x9312e55 0x9312f33 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:36.959 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0xdd2f7 0x4ab1c 0x3ac7302 0x3b8c153 0xb9cc43a 0xb9cd8da 0xb9cdb54 0xb9cdcfc 0xb9e567f 0xba14771 0xba14db8 0xb9bc06f 0xb9bfa98 0xb9aaa36 0xb9aae1a 0x9add3c3 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x931138d 0x931232c 0x93123af 0x9312434 0x9312536 0x931262e 0x93126b4 0x931276d 0x9312923 0x9312a5b 0x9312ad6 0x9312b5b 0x9312cb6 0x9312dd0 0x9312e55 0x9312f33 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:36.959 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:36.960 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:37.017 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 90 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0xba0b97a 0xba0ba6f 0xb9bb43d 0xb9bfa98 0xb9aaa36 0xb9aae1a 0x9df53d6 0x9e0571e 0x99e3351 0x99e3957 0x98d8890 0x9311591 0x931232c 0x93123af 0x9312434 0x9312536 0x931262e 0x93126b4 0x931276d 0x9312923 0x9312a5b 0x9312ad6 0x9312b5b 0x9312cb6 0x9312dd0 0x9312e55 0x9312f33 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:37.017 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:37.017 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 90 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x36dd504 0x36e08d7 0x36e0d04 0x36e11ca 0x36e1614 0xc4a96ec 0xc49545e 0x9e20876 0x9df83e9 0x9e054a3 0x99e3351 0x99e3957 0x98d8890 0x9311591 0x931232c 0x93123af 0x9312434 0x9312536 0x931262e 0x93126b4 0x931276d 0x9312923 0x9312a5b 0x9312ad6 0x9312b5b 0x9312cb6 0x9312dd0 0x9312e55 0x9312f33 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:37.017 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:37.018 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x156cf 0xdd312 0x4bedc 0xb9b1558 0x9826b31 0x9adcd14 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x9258152 0x925866e 0x92587f7 0x9258a3e 0x9258d16 0x9258dcb 0x9576088 0x95762ed 0x95ae2d3 0x96b28be 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:37.018 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:37.076 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 123 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0xdd323 0x4ab1c 0x5c73334 0xb9cdb42 0xb9cdcfc 0xb9cfab7 0xb9cfcb3 0xba14b9d 0xb9bc06f 0xb9bfa98 0xb9aaa36 0xb9aae1a 0x9add3c3 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x9258152 0x925866e 0x92587f7 0x9258a3e 0x9258d16 0x9258dcb 0x9576088 0x95762ed 0x95ae2d3 0x96b28be 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:37.076 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:37.205 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x9e16d06 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:37.205 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:37.263 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 122 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x5525882 0x9e16d1b 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:37.263 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:37.299 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x133cc6 0xdd332 0x4ab1c 0x5525eb7 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:37.299 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:37.300 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x8b55f 0x10d76c 0xdd31a 0x4ab1c 0x5525eb7 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:37.301 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:37.357 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 122 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x4ab0c 0x5525eb7 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:37.358 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:37.358 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 122 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x552576d 0x9e16d1b 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:37.358 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:37.370 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 230 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x9e16cc3 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:37.370 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:37.671 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x4ab7c 0xb9cc4a3 0xb9cd8da 0xb9cdb54 0xb9cdcfc 0xb9e53f4 0xba14771 0xba14db8 0xb9bc06f 0xb9bfa98 0xb9aaa36 0xb9aae1a 0x9add3c3 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x92f52b6 0x92f5d5a 0x92f5ddd 0x92f5e62 0x92f5f64 0x92f605c 0x92f60e2 0x92f619b 0x92f6351 0x92f6489 0x92f6504 0x92f6589 0x92f66e4 0x92f67fe 0x92f6883 0x92f6961 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:37.671 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x156e8 0xdd312 0x4b14c 0x3b8bfdd 0x3b8c15e 0xb9d16cf 0xb9d2155 0xba1750a 0xb9b1eb4 0x9826b31 0x9add2fa 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x92f52b6 0x92f5d5a 0x92f5ddd 0x92f5e62 0x92f5f64 0x92f605c 0x92f60e2 0x92f619b 0x92f6351 0x92f6489 0x92f6504 0x92f6589 0x92f66e4 0x92f67fe 0x92f6883 0x92f6961 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:37.671 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:37.671 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:38.761 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-22T03:46:38.761 INFO:teuthology.orchestra.run.smithi123.stderr:Suppressions used: 2024-09-22T03:46:38.762 INFO:teuthology.orchestra.run.smithi123.stderr: count bytes template 2024-09-22T03:46:38.762 INFO:teuthology.orchestra.run.smithi123.stderr: 1 8 MallocExtension::Register 2024-09-22T03:46:38.762 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-22T03:46:38.762 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-22T03:46:38.787 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-22T03:46:38.812 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:46:39.091 INFO:teuthology.orchestra.run.smithi123.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:46:39.092 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-22 03:46:39,092 seastar - Reactor backend: linux-aio 2024-09-22T03:46:39.097 INFO:teuthology.orchestra.run.smithi123.stderr:==27671==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:46:39.217 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-22 03:46:39,217 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-22T03:46:39.217 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-22 03:46:39,217 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 3-5 2024-09-22T03:46:39.218 INFO:teuthology.orchestra.run.smithi123.stderr:DEBUG 2024-09-22 03:46:39,218 [shard 0:main] seastar - reactor::drain 2024-09-22T03:46:39.715 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-22T03:46:39.715 INFO:teuthology.orchestra.run.smithi123.stderr:Suppressions used: 2024-09-22T03:46:39.715 INFO:teuthology.orchestra.run.smithi123.stderr: count bytes template 2024-09-22T03:46:39.715 INFO:teuthology.orchestra.run.smithi123.stderr: 1 8 MallocExtension::Register 2024-09-22T03:46:39.715 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-22T03:46:39.715 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-22T03:46:39.773 INFO:teuthology.orchestra.run.smithi123.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-22T03:46:39.777 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-22 03:46:39,776 seastar - Reactor backend: linux-aio 2024-09-22T03:46:39.792 INFO:teuthology.orchestra.run.smithi123.stderr:==27669==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-22T03:46:40.074 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0xc9232 0xc9e39 0x36074e8 0x3b8bf66 0x3b8c15e 0xb9cc607 0xb9cd8da 0xb9cdb54 0xb9cdcfc 0xb9cfab7 0xb9cff02 0xb9bbe2c 0xb9bfa98 0xb9aaa36 0xb9aae1a 0x9add3c3 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x931138d 0x931232c 0x93123af 0x9312434 0x9312536 0x931262e 0x93126b4 0x931276d 0x9312923 0x9312a5b 0x9312ad6 0x9312b5b 0x9312cb6 0x9312dd0 0x9312e55 0x9312f33 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:40.074 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0xc95d6 0xc9e39 0x36074e8 0x3b8bf66 0x3b8c15e 0xb9cc607 0xb9cd8da 0xb9cdb54 0xb9cdcfc 0xb9cfab7 0xb9cff02 0xb9bbe2c 0xb9bfa98 0xb9aaa36 0xb9aae1a 0x9add3c3 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x931138d 0x931232c 0x93123af 0x9312434 0x9312536 0x931262e 0x93126b4 0x931276d 0x9312923 0x9312a5b 0x9312ad6 0x9312b5b 0x9312cb6 0x9312dd0 0x9312e55 0x9312f33 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:40.074 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:40.074 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:40.132 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 90 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x36e1196 0x36e1614 0xc4b3528 0xc4975cb 0xc4954df 0xb0b6299 0xb0b0970 0xac78421 0xacde0f1 0xacab3b6 0x99296ea 0x9257a3d 0x931138d 0x931232c 0x93123af 0x9312434 0x9312536 0x931262e 0x93126b4 0x931276d 0x9312923 0x9312a5b 0x9312ad6 0x9312b5b 0x9312cb6 0x9312dd0 0x9312e55 0x9312f33 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:40.132 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 90 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x3610484 0xba0a92a 0xba0ade1 0xb9aef29 0xba0b778 0xba0b9bf 0xba0ba6f 0xb9bb43d 0xb9bfa98 0xb9aaa36 0xb9aae1a 0x9df53d6 0x9e0571e 0x99e3351 0x99e3957 0x98d8890 0x9311591 0x931232c 0x93123af 0x9312434 0x9312536 0x931262e 0x93126b4 0x931276d 0x9312923 0x9312a5b 0x9312ad6 0x9312b5b 0x9312cb6 0x9312dd0 0x9312e55 0x9312f33 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:40.132 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:40.132 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:40.158 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0xb9d1801 0xb9b215a 0x9826b31 0x9adcd14 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x9258152 0x925866e 0x92587f7 0x9258a3e 0x9258d16 0x9258dcb 0x9576088 0x95762ed 0x95ae2d3 0x96b28be 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:40.158 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:40.330 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0xdd323 0x4ab1c 0x5525eb7 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:40.330 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:40.388 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 122 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0xdd332 0x4ab1c 0x5525eb7 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:40.388 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:40.430 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x55253cb 0x9e16d0e 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:40.430 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:40.438 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x133ccf 0xdd332 0x4ab1c 0x5525eb7 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:40.438 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:40.487 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 122 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x5525e29 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:40.487 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:40.495 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 230 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x55251ec 0x9e16cf9 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:40.495 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:40.496 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 122 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0xdd31f 0x4ab1c 0x5525eb7 0x9e16ce0 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:40.496 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:40.798 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0xb83633f 0xb83655a 0xb836643 0xb836903 0xb8369f6 0xb836ad6 0xb837196 0xb837301 0xb83770c 0xb8379cb 0xba13cb0 0xba13f4c 0xba14113 0xba14387 0xba14490 0xba1460c 0xba14db8 0xb9bc06f 0xb9bfa98 0xb9aaa36 0xb9aae1a 0x9add3c3 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x92f52b6 0x92f5d5a 0x92f5ddd 0x92f5e62 0x92f5f64 0x92f605c 0x92f60e2 0x92f619b 0x92f6351 0x92f6489 0x92f6504 0x92f6589 0x92f66e4 0x92f67fe 0x92f6883 0x92f6961 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:40.798 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x444f4 0xfe5da 0x5ca6522 0x5ca66fc 0x5ca69b3 0x5ca6978 0x5ca6aa8 0x961a19c 0x9add627 0x9af4f12 0x9985508 0x9925cf1 0x9257a3d 0x92f52b6 0x92f5d5a 0x92f5ddd 0x92f5e62 0x92f5f64 0x92f605c 0x92f60e2 0x92f619b 0x92f6351 0x92f6489 0x92f6504 0x92f6589 0x92f66e4 0x92f67fe 0x92f6883 0x92f6961 0x9535c26 0x95fe993 0x95feb10 0x96bf871 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc5034b 0xbc509cd 0x7b13bb5 0xc5e60e4 0x5e669 0x89c51 0x10ec7f 2024-09-22T03:46:40.798 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:40.798 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:40.932 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x9e16cc3 0x9e861d9 0x9eb69d1 0x9e1df64 0x993c4f7 0x985b902 0x987ffc2 0x9880121 0x98919be 0x9891b8e 0x9891e9c 0x98921ab 0x989230a 0x9892385 0x98925cc 0x9892890 0x9892920 0x98929c4 0x95a6852 0x95a6ab7 0x95af05c 0x96a1706 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-22T03:46:40.933 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-22T03:46:41.881 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-22T03:46:41.881 INFO:teuthology.orchestra.run.smithi123.stderr:Suppressions used: 2024-09-22T03:46:41.881 INFO:teuthology.orchestra.run.smithi123.stderr: count bytes template 2024-09-22T03:46:41.881 INFO:teuthology.orchestra.run.smithi123.stderr: 1 8 MallocExtension::Register 2024-09-22T03:46:41.881 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-22T03:46:41.881 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-22T03:46:41.907 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-22T03:46:41.931 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-22T03:46:41.932 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:46:41.932 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-22T03:46:41.963 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:46:41.963 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-22T03:46:42.033 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-22T03:46:42.033 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-22T03:46:42.057 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-22T03:46:42.057 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-22T03:46:42.121 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-22T03:46:42.121 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-22T03:46:42.184 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:46:42.185 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-22T03:46:42.205 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-22T03:46:42.205 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-22T03:46:42.240 INFO:tasks.ceph:Adding keys to all mons... 2024-09-22T03:46:42.241 DEBUG:teuthology.orchestra.run.smithi003:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-22T03:46:42.271 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.0] 2024-09-22T03:46:42.271 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAUk+9mPgXMKRAAzQEide8dZdIghTQNc8K6DA== 2024-09-22T03:46:42.271 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.1] 2024-09-22T03:46:42.271 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAXk+9mqbyNOBAAtttwdLdO5/JfnZpmlm3ahA== 2024-09-22T03:46:42.271 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.x] 2024-09-22T03:46:42.271 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQARk+9mKj1iMxAA/wE1eOV6vnsZmaME/OKyfg== 2024-09-22T03:46:42.271 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.2] 2024-09-22T03:46:42.271 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAck+9mkNj9MBAAAsyKo6KrF78WpUVtMlAcYg== 2024-09-22T03:46:42.271 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.3] 2024-09-22T03:46:42.271 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAfk+9mF3XQNxAASsAUXbK2AX2wtqU+aAM1FQ== 2024-09-22T03:46:42.272 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-22T03:46:42.272 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQARk+9m/0SpNhAArtk/fFNvap6Z42YZli5zHQ== 2024-09-22T03:46:42.272 INFO:teuthology.orchestra.run.smithi003.stdout:[client.1] 2024-09-22T03:46:42.272 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQARk+9mQlRvOhAA3eWTrE5fDtdJTyvPTAsBPA== 2024-09-22T03:46:42.273 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T03:46:42.386 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T03:46:42.451 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-22T03:46:42.513 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T03:46:42.576 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T03:46:42.639 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-22T03:46:42.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-22T03:46:42.770 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-22T03:46:42.771 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-22T03:46:42.797 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-22T03:46:42.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:42.916+0000 7f98456d4c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:42.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:42.916+0000 7f98456d4c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:42.930 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-22T03:46:42.956 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-22T03:46:43.011 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-22T03:46:43.012 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-22T03:46:43.012 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-22T03:46:43.055 INFO:tasks.ceph.mon.a:Started 2024-09-22T03:46:43.055 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-22T03:46:43.055 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-22T03:46:43.055 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-22T03:46:43.057 INFO:tasks.ceph.mgr.x:Started 2024-09-22T03:46:43.057 DEBUG:tasks.ceph:set 0 configs 2024-09-22T03:46:43.057 INFO:tasks.ceph:Setting crush tunables to default 2024-09-22T03:46:43.058 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd crush tunables default 2024-09-22T03:46:43.139 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-22T03:46:43.138+0000 7fe815435c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:43.140 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-22T03:46:43.138+0000 7fe815435c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:43.140 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-22T03:46:43.138+0000 7fe815435c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:43.153 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-22T03:46:43.152+0000 7f0372bcc000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:43.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.161+0000 7f4afa09c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:43.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.168+0000 7f4afa09c640 1 Processor -- start 2024-09-22T03:46:43.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.169+0000 7f4afa09c640 1 -- start start 2024-09-22T03:46:43.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.170+0000 7f4afa09c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4af4073d40 0x7f4af413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:43.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.170+0000 7f4afa09c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4af40a2690 con 0x7f4af4073220 2024-09-22T03:46:43.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.170+0000 7f4afa09c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4af40a10a0 con 0x7f4af4073d40 2024-09-22T03:46:43.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.170+0000 7f4af2ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4af4073d40 0x7f4af413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:43.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.170+0000 7f4af37fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4af4073220 0x7f4af4073630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45634/0 (socket says 172.21.15.3:45634) 2024-09-22T03:46:43.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.170+0000 7f4af37fe640 1 -- 172.21.15.3:0/19932199 learned_addr learned my addr 172.21.15.3:0/19932199 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T03:46:43.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.170+0000 7f4af2ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4af4073d40 0x7f4af413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46384/0 (socket says 172.21.15.3:46384) 2024-09-22T03:46:43.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.170+0000 7f4af2ffd640 1 -- 172.21.15.3:0/19932199 >> v2:172.21.15.3:3300/0 conn(0x7f4af4073d40 msgr2=0x7f4af413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-22T03:46:43.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.170+0000 7f4af2ffd640 1 -- 172.21.15.3:0/19932199 >> v2:172.21.15.3:3300/0 conn(0x7f4af4073d40 msgr2=0x7f4af413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-22T03:46:43.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.170+0000 7f4af2ffd640 1 --2- 172.21.15.3:0/19932199 >> v2:172.21.15.3:3300/0 conn(0x7f4af4073d40 0x7f4af413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-22T03:46:43.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.170+0000 7f4af2ffd640 1 --2- 172.21.15.3:0/19932199 >> v2:172.21.15.3:3300/0 conn(0x7f4af4073d40 0x7f4af413b160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-22T03:46:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.174+0000 7f4af27fc640 1 -- 172.21.15.3:0/19932199 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1007044783 0 0) 0x7f4af40a2690 con 0x7f4af4073220 2024-09-22T03:46:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.174+0000 7f4af27fc640 1 -- 172.21.15.3:0/19932199 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4adc003580 con 0x7f4af4073220 2024-09-22T03:46:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.174+0000 7f4af27fc640 1 -- 172.21.15.3:0/19932199 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 46840446 0 0) 0x7f4adc003580 con 0x7f4af4073220 2024-09-22T03:46:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.174+0000 7f4af27fc640 1 -- 172.21.15.3:0/19932199 >> v2:172.21.15.3:3300/0 conn(0x7f4af4073d40 msgr2=0x7f4af413b160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-22T03:46:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.174+0000 7f4af27fc640 1 --2- 172.21.15.3:0/19932199 >> v2:172.21.15.3:3300/0 conn(0x7f4af4073d40 0x7f4af413b160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.174+0000 7f4af27fc640 1 -- 172.21.15.3:0/19932199 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af40a2380 con 0x7f4af4073220 2024-09-22T03:46:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.174+0000 7f4af27fc640 1 -- 172.21.15.3:0/19932199 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 205+0+0 (unknown 2914378549 0 0) 0x7f4ae4001620 con 0x7f4af4073220 2024-09-22T03:46:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.174+0000 7f4af27fc640 1 -- 172.21.15.3:0/19932199 >> v1:172.21.15.3:6789/0 conn(0x7f4af4073220 legacy=0x7f4af4073630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.174+0000 7f4af27fc640 1 --2- 172.21.15.3:0/19932199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc003a70 0x7f4adc003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.174+0000 7f4af27fc640 1 -- 172.21.15.3:0/19932199 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4af40a10a0 con 0x7f4adc003a70 2024-09-22T03:46:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.174+0000 7f4af37fe640 1 --2- 172.21.15.3:0/19932199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc003a70 0x7f4adc003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.174+0000 7f4af37fe640 1 -- 172.21.15.3:0/19932199 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4adc003580 con 0x7f4adc003a70 2024-09-22T03:46:43.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.176+0000 7f4af37fe640 1 --2- 172.21.15.3:0/19932199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc003a70 0x7f4adc003ec0 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee84081e826e596 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:43.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.176+0000 7f4af27fc640 1 -- 172.21.15.3:0/19932199 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4ae4007cc0 con 0x7f4adc003a70 2024-09-22T03:46:43.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.176+0000 7f4af27fc640 1 -- 172.21.15.3:0/19932199 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4ae4005930 con 0x7f4adc003a70 2024-09-22T03:46:43.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.176+0000 7f4af27fc640 1 -- 172.21.15.3:0/19932199 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4ae4005bd0 con 0x7f4adc003a70 2024-09-22T03:46:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.177+0000 7f4afa09c640 1 -- 172.21.15.3:0/19932199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc003a70 msgr2=0x7f4adc003ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.177+0000 7f4afa09c640 1 --2- 172.21.15.3:0/19932199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc003a70 0x7f4adc003ec0 crc :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.177+0000 7f4afa09c640 1 -- 172.21.15.3:0/19932199 shutdown_connections 2024-09-22T03:46:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.177+0000 7f4afa09c640 1 --2- 172.21.15.3:0/19932199 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc003a70 0x7f4adc003ec0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.177+0000 7f4afa09c640 1 --2- 172.21.15.3:0/19932199 >> v2:172.21.15.3:3300/0 conn(0x7f4af4073d40 0x7f4af413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.177+0000 7f4afa09c640 1 -- 172.21.15.3:0/19932199 >> 172.21.15.3:0/19932199 conn(0x7f4af406f6e0 msgr2=0x7f4af4071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.177+0000 7f4afa09c640 1 -- 172.21.15.3:0/19932199 shutdown_connections 2024-09-22T03:46:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.177+0000 7f4afa09c640 1 -- 172.21.15.3:0/19932199 wait complete. 2024-09-22T03:46:43.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.178+0000 7f4afa09c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:43.180 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-22T03:46:43.178+0000 7f0372bcc000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:43.180 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-22T03:46:43.178+0000 7f0372bcc000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:43.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.179+0000 7f4afa09c640 1 Processor -- start 2024-09-22T03:46:43.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.180+0000 7f4afa09c640 1 -- start start 2024-09-22T03:46:43.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.181+0000 7f4afa09c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc003a70 0x7f4af4136800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.181+0000 7f4afa09c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4af4126be0 con 0x7f4adc003a70 2024-09-22T03:46:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.181+0000 7f4af37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc003a70 0x7f4af4136800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.181+0000 7f4af37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc003a70 0x7f4af4136800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46396/0 (socket says 172.21.15.3:46396) 2024-09-22T03:46:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.181+0000 7f4af37fe640 1 -- 172.21.15.3:0/2440056565 learned_addr learned my addr 172.21.15.3:0/2440056565 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.182+0000 7f4af37fe640 1 -- 172.21.15.3:0/2440056565 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af4129880 con 0x7f4adc003a70 2024-09-22T03:46:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.182+0000 7f4af37fe640 1 --2- 172.21.15.3:0/2440056565 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc003a70 0x7f4af4136800 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.182+0000 7f4adbfff640 1 -- 172.21.15.3:0/2440056565 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4ae4001080 con 0x7f4adc003a70 2024-09-22T03:46:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.182+0000 7f4adbfff640 1 -- 172.21.15.3:0/2440056565 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4ae4007250 con 0x7f4adc003a70 2024-09-22T03:46:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.182+0000 7f4afa09c640 1 -- 172.21.15.3:0/2440056565 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af4127320 con 0x7f4adc003a70 2024-09-22T03:46:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.182+0000 7f4adbfff640 1 -- 172.21.15.3:0/2440056565 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4ae40074f0 con 0x7f4adc003a70 2024-09-22T03:46:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.182+0000 7f4afa09c640 1 -- 172.21.15.3:0/2440056565 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af4128cc0 con 0x7f4adc003a70 2024-09-22T03:46:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.182+0000 7f4adbfff640 1 -- 172.21.15.3:0/2440056565 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f4ae40076a0 con 0x7f4adc003a70 2024-09-22T03:46:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.182+0000 7f4adbfff640 1 -- 172.21.15.3:0/2440056565 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (crc 0 0 0) 0x7f4ae4015d30 con 0x7f4adc003a70 2024-09-22T03:46:43.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.183+0000 7f4afa09c640 1 -- 172.21.15.3:0/2440056565 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af406a460 con 0x7f4adc003a70 2024-09-22T03:46:43.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.185+0000 7f4adbfff640 1 -- 172.21.15.3:0/2440056565 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (crc 0 0 0) 0x7f4ae40153b0 con 0x7f4adc003a70 2024-09-22T03:46:43.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.250+0000 7f4afa09c640 1 -- 172.21.15.3:0/2440056565 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f4af4077360 con 0x7f4adc003a70 2024-09-22T03:46:43.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.251+0000 7f4adbfff640 1 -- 172.21.15.3:0/2440056565 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (crc 0 0 0) 0x7f4ae4015550 con 0x7f4adc003a70 2024-09-22T03:46:43.252 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2024-09-22T03:46:43.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.252+0000 7f4afa09c640 1 -- 172.21.15.3:0/2440056565 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc003a70 msgr2=0x7f4af4136800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:43.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.252+0000 7f4afa09c640 1 --2- 172.21.15.3:0/2440056565 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc003a70 0x7f4af4136800 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:43.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.252+0000 7f4afa09c640 1 -- 172.21.15.3:0/2440056565 shutdown_connections 2024-09-22T03:46:43.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.252+0000 7f4afa09c640 1 --2- 172.21.15.3:0/2440056565 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4adc003a70 0x7f4af4136800 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:43.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.252+0000 7f4afa09c640 1 -- 172.21.15.3:0/2440056565 >> 172.21.15.3:0/2440056565 conn(0x7f4af406f6e0 msgr2=0x7f4af411eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:43.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.252+0000 7f4afa09c640 1 -- 172.21.15.3:0/2440056565 shutdown_connections 2024-09-22T03:46:43.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.252+0000 7f4afa09c640 1 -- 172.21.15.3:0/2440056565 wait complete. 2024-09-22T03:46:43.264 INFO:tasks.ceph:check_enable_crimson: True 2024-09-22T03:46:43.264 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2024-09-22T03:46:43.265 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2024-09-22T03:46:43.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.368+0000 7fbe220d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:43.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.374+0000 7fbe220d1640 1 Processor -- start 2024-09-22T03:46:43.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.374+0000 7fbe220d1640 1 -- start start 2024-09-22T03:46:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.376+0000 7fbe220d1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbe1c073d10 0x7fbe1c138fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.376+0000 7fbe220d1640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe1c09d940 con 0x7fbe1c0731f0 2024-09-22T03:46:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.376+0000 7fbe220d1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbe1c09d1c0 con 0x7fbe1c073d10 2024-09-22T03:46:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.376+0000 7fbe1b7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbe1c0731f0 0x7fbe1c073600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45648/0 (socket says 172.21.15.3:45648) 2024-09-22T03:46:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.376+0000 7fbe1b7fe640 1 -- 172.21.15.3:0/843099666 learned_addr learned my addr 172.21.15.3:0/843099666 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T03:46:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.376+0000 7fbe1affd640 1 --2- 172.21.15.3:0/843099666 >> v2:172.21.15.3:3300/0 conn(0x7fbe1c073d10 0x7fbe1c138fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.376+0000 7fbe1a7fc640 1 -- 172.21.15.3:0/843099666 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1604653299 0 0) 0x7fbe1c09d940 con 0x7fbe1c0731f0 2024-09-22T03:46:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.376+0000 7fbe1a7fc640 1 -- 172.21.15.3:0/843099666 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe04003580 con 0x7fbe1c0731f0 2024-09-22T03:46:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.376+0000 7fbe1affd640 1 -- 172.21.15.3:0/843099666 >> v1:172.21.15.3:6789/0 conn(0x7fbe1c0731f0 legacy=0x7fbe1c073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.376+0000 7fbe1affd640 1 -- 172.21.15.3:0/843099666 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1c09e220 con 0x7fbe1c073d10 2024-09-22T03:46:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.377+0000 7fbe1affd640 1 --2- 172.21.15.3:0/843099666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe1c073d10 0x7fbe1c138fc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7efc2f8bc6121 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.377+0000 7fbe1a7fc640 1 -- 172.21.15.3:0/843099666 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbe10009d90 con 0x7fbe1c073d10 2024-09-22T03:46:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.377+0000 7fbe1a7fc640 1 -- 172.21.15.3:0/843099666 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbe1000e230 con 0x7fbe1c073d10 2024-09-22T03:46:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.377+0000 7fbe1a7fc640 1 -- 172.21.15.3:0/843099666 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbe1000e4d0 con 0x7fbe1c073d10 2024-09-22T03:46:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.377+0000 7fbe220d1640 1 -- 172.21.15.3:0/843099666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe1c073d10 msgr2=0x7fbe1c138fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.377+0000 7fbe220d1640 1 --2- 172.21.15.3:0/843099666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe1c073d10 0x7fbe1c138fc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.377+0000 7fbe220d1640 1 -- 172.21.15.3:0/843099666 shutdown_connections 2024-09-22T03:46:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.377+0000 7fbe220d1640 1 --2- 172.21.15.3:0/843099666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe1c073d10 0x7fbe1c138fc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.377+0000 7fbe220d1640 1 -- 172.21.15.3:0/843099666 >> 172.21.15.3:0/843099666 conn(0x7fbe1c0a6d10 msgr2=0x7fbe1c071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.377+0000 7fbe220d1640 1 -- 172.21.15.3:0/843099666 shutdown_connections 2024-09-22T03:46:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.378+0000 7fbe220d1640 1 -- 172.21.15.3:0/843099666 wait complete. 2024-09-22T03:46:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.378+0000 7fbe220d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.379+0000 7fbe220d1640 1 Processor -- start 2024-09-22T03:46:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.380+0000 7fbe220d1640 1 -- start start 2024-09-22T03:46:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.381+0000 7fbe220d1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe1c0731f0 0x7fbe1c06f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.381+0000 7fbe220d1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c126570 con 0x7fbe1c0731f0 2024-09-22T03:46:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.381+0000 7fbe1b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe1c0731f0 0x7fbe1c06f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.381+0000 7fbe1b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe1c0731f0 0x7fbe1c06f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46412/0 (socket says 172.21.15.3:46412) 2024-09-22T03:46:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.381+0000 7fbe1b7fe640 1 -- 172.21.15.3:0/3696547690 learned_addr learned my addr 172.21.15.3:0/3696547690 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.381+0000 7fbe1b7fe640 1 -- 172.21.15.3:0/3696547690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1c127bd0 con 0x7fbe1c0731f0 2024-09-22T03:46:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.382+0000 7fbe1b7fe640 1 --2- 172.21.15.3:0/3696547690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe1c0731f0 0x7fbe1c06f8c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.382+0000 7fbe03fff640 1 -- 172.21.15.3:0/3696547690 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbe0c005a60 con 0x7fbe1c0731f0 2024-09-22T03:46:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.382+0000 7fbe03fff640 1 -- 172.21.15.3:0/3696547690 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbe0c005bc0 con 0x7fbe1c0731f0 2024-09-22T03:46:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.382+0000 7fbe220d1640 1 -- 172.21.15.3:0/3696547690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1c071330 con 0x7fbe1c0731f0 2024-09-22T03:46:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.382+0000 7fbe03fff640 1 -- 172.21.15.3:0/3696547690 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbe0c005e60 con 0x7fbe1c0731f0 2024-09-22T03:46:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.382+0000 7fbe220d1640 1 -- 172.21.15.3:0/3696547690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe1c125f90 con 0x7fbe1c0731f0 2024-09-22T03:46:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.382+0000 7fbe03fff640 1 -- 172.21.15.3:0/3696547690 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7fbe0c007440 con 0x7fbe1c0731f0 2024-09-22T03:46:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.382+0000 7fbe03fff640 1 -- 172.21.15.3:0/3696547690 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (crc 0 0 0) 0x7fbe0c00f8e0 con 0x7fbe1c0731f0 2024-09-22T03:46:43.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.383+0000 7fbe220d1640 1 -- 172.21.15.3:0/3696547690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe1c09eda0 con 0x7fbe1c0731f0 2024-09-22T03:46:43.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.385+0000 7fbe03fff640 1 -- 172.21.15.3:0/3696547690 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (crc 0 0 0) 0x7fbe0c00faa0 con 0x7fbe1c0731f0 2024-09-22T03:46:43.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:43.460+0000 7fbe220d1640 1 -- 172.21.15.3:0/3696547690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true} v 0) -- 0x7fbe1c073600 con 0x7fbe1c0731f0 2024-09-22T03:46:43.461 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-22T03:46:43.460+0000 7fe80bdf4640 -1 WARNING: experimental feature 'crimson' is enabled 2024-09-22T03:46:43.461 INFO:tasks.ceph.mon.a.smithi003.stderr:Please be aware that this feature is experimental, untested, 2024-09-22T03:46:43.462 INFO:tasks.ceph.mon.a.smithi003.stderr:unsupported, and may result in data corruption, data loss, 2024-09-22T03:46:43.462 INFO:tasks.ceph.mon.a.smithi003.stderr:and/or irreparable damage to your cluster. Do not use 2024-09-22T03:46:43.462 INFO:tasks.ceph.mon.a.smithi003.stderr:feature with important data. 2024-09-22T03:46:43.462 INFO:tasks.ceph.mon.a.smithi003.stderr: 2024-09-22T03:46:44.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:44.252+0000 7fbe03fff640 1 -- 172.21.15.3:0/3696547690 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd set-allow-crimson", "yes_i_really_mean_it": true}]=0 v3) ==== 99+0+0 (crc 0 0 0) 0x7fbe0c00fc40 con 0x7fbe1c0731f0 2024-09-22T03:46:44.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:44.253+0000 7fbe220d1640 1 -- 172.21.15.3:0/3696547690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe1c0731f0 msgr2=0x7fbe1c06f8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:44.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:44.253+0000 7fbe220d1640 1 --2- 172.21.15.3:0/3696547690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe1c0731f0 0x7fbe1c06f8c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:44.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:44.253+0000 7fbe220d1640 1 -- 172.21.15.3:0/3696547690 shutdown_connections 2024-09-22T03:46:44.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:44.253+0000 7fbe220d1640 1 --2- 172.21.15.3:0/3696547690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe1c0731f0 0x7fbe1c06f8c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:44.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:44.253+0000 7fbe220d1640 1 -- 172.21.15.3:0/3696547690 >> 172.21.15.3:0/3696547690 conn(0x7fbe1c0a6d10 msgr2=0x7fbe1c0a4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:44.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:44.254+0000 7fbe220d1640 1 -- 172.21.15.3:0/3696547690 shutdown_connections 2024-09-22T03:46:44.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:44.254+0000 7fbe220d1640 1 -- 172.21.15.3:0/3696547690 wait complete. 2024-09-22T03:46:44.269 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-22T03:46:44.269 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:46:44.270 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-22T03:46:44.299 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:46:44.299 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-22T03:46:44.369 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-22T03:46:44.369 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-22T03:46:44.394 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-22T03:46:44.394 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-22T03:46:44.459 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new ae27f46d-f8f7-4336-ad5e-ef956da82898 0 2024-09-22T03:46:44.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.599+0000 7f3e1b0fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:44.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.608+0000 7f3e1b0fd640 1 Processor -- start 2024-09-22T03:46:44.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.608+0000 7f3e1b0fd640 1 -- start start 2024-09-22T03:46:44.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.608+0000 7f3e1b0fd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3e141199a0 0x7f3e14119db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:44.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.608+0000 7f3e1b0fd640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e14115870 con 0x7f3e14114d50 2024-09-22T03:46:44.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.608+0000 7f3e1b0fd640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3e14115a00 con 0x7f3e141199a0 2024-09-22T03:46:44.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.609+0000 7f3e18e99640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3e141199a0 0x7f3e14119db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:44.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.609+0000 7f3e13fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3e14114d50 0x7f3e14115160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.123:59960/0 (socket says 172.21.15.123:59960) 2024-09-22T03:46:44.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.609+0000 7f3e13fff640 1 -- 172.21.15.123:0/2344654913 learned_addr learned my addr 172.21.15.123:0/2344654913 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-09-22T03:46:44.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.609+0000 7f3e18e99640 1 -- 172.21.15.123:0/2344654913 >> v1:172.21.15.3:6789/0 conn(0x7f3e14114d50 legacy=0x7f3e14115160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:44.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.609+0000 7f3e18e99640 1 -- 172.21.15.123:0/2344654913 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e14115b70 con 0x7f3e141199a0 2024-09-22T03:46:44.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.609+0000 7f3e18e99640 1 --2- 172.21.15.123:0/2344654913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e141199a0 0x7f3e14119db0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d89bc61a5381149 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:44.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.609+0000 7f3e137fe640 1 -- 172.21.15.123:0/2344654913 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3e08009d90 con 0x7f3e141199a0 2024-09-22T03:46:44.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.610+0000 7f3e137fe640 1 -- 172.21.15.123:0/2344654913 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3e0800e230 con 0x7f3e141199a0 2024-09-22T03:46:44.611 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.610+0000 7f3e137fe640 1 -- 172.21.15.123:0/2344654913 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3e0800e4d0 con 0x7f3e141199a0 2024-09-22T03:46:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.610+0000 7f3e1b0fd640 1 -- 172.21.15.123:0/2344654913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e141199a0 msgr2=0x7f3e14119db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.610+0000 7f3e1b0fd640 1 --2- 172.21.15.123:0/2344654913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e141199a0 0x7f3e14119db0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.610+0000 7f3e1b0fd640 1 -- 172.21.15.123:0/2344654913 shutdown_connections 2024-09-22T03:46:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.610+0000 7f3e1b0fd640 1 --2- 172.21.15.123:0/2344654913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e141199a0 0x7f3e14119db0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.610+0000 7f3e1b0fd640 1 -- 172.21.15.123:0/2344654913 >> 172.21.15.123:0/2344654913 conn(0x7f3e1406fab0 msgr2=0x7f3e1406fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:44.612 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.611+0000 7f3e1b0fd640 1 -- 172.21.15.123:0/2344654913 shutdown_connections 2024-09-22T03:46:44.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.611+0000 7f3e1b0fd640 1 -- 172.21.15.123:0/2344654913 wait complete. 2024-09-22T03:46:44.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.611+0000 7f3e1b0fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:44.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.611+0000 7f3e1b0fd640 1 Processor -- start 2024-09-22T03:46:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.611+0000 7f3e1b0fd640 1 -- start start 2024-09-22T03:46:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.612+0000 7f3e1b0fd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e14114d50 0x7f3e141a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.612+0000 7f3e1b0fd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141a9e80 con 0x7f3e14114d50 2024-09-22T03:46:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.612+0000 7f3e18e99640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e14114d50 0x7f3e141a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.612+0000 7f3e18e99640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e14114d50 0x7f3e141a98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.123:52022/0 (socket says 172.21.15.123:52022) 2024-09-22T03:46:44.614 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.612+0000 7f3e18e99640 1 -- 172.21.15.123:0/896530139 learned_addr learned my addr 172.21.15.123:0/896530139 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-22T03:46:44.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.613+0000 7f3e18e99640 1 -- 172.21.15.123:0/896530139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e141a9fc0 con 0x7f3e14114d50 2024-09-22T03:46:44.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.613+0000 7f3e18e99640 1 --2- 172.21.15.123:0/896530139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e14114d50 0x7f3e141a98b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:44.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.613+0000 7f3e117fa640 1 -- 172.21.15.123:0/896530139 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3e08014620 con 0x7f3e14114d50 2024-09-22T03:46:44.615 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.613+0000 7f3e117fa640 1 -- 172.21.15.123:0/896530139 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3e08014780 con 0x7f3e14114d50 2024-09-22T03:46:44.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.613+0000 7f3e1b0fd640 1 -- 172.21.15.123:0/896530139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e141aa210 con 0x7f3e14114d50 2024-09-22T03:46:44.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.613+0000 7f3e117fa640 1 -- 172.21.15.123:0/896530139 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3e08014a20 con 0x7f3e14114d50 2024-09-22T03:46:44.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.613+0000 7f3e1b0fd640 1 -- 172.21.15.123:0/896530139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e141af110 con 0x7f3e14114d50 2024-09-22T03:46:44.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.613+0000 7f3e117fa640 1 -- 172.21.15.123:0/896530139 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f3e08013400 con 0x7f3e14114d50 2024-09-22T03:46:44.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.614+0000 7f3e117fa640 1 -- 172.21.15.123:0/896530139 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 747+0+0 (crc 0 0 0) 0x7f3e08019040 con 0x7f3e14114d50 2024-09-22T03:46:44.616 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.614+0000 7f3e1b0fd640 1 -- 172.21.15.123:0/896530139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e14120ba0 con 0x7f3e14114d50 2024-09-22T03:46:44.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.617+0000 7f3e117fa640 1 -- 172.21.15.123:0/896530139 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (crc 0 0 0) 0x7f3e08013890 con 0x7f3e14114d50 2024-09-22T03:46:44.689 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.688+0000 7f3e1b0fd640 1 -- 172.21.15.123:0/896530139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ae27f46d-f8f7-4336-ad5e-ef956da82898", "id": 0} v 0) -- 0x7f3e14120d80 con 0x7f3e14114d50 2024-09-22T03:46:44.691 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.690+0000 7f3e117fa640 1 -- 172.21.15.123:0/896530139 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ae27f46d-f8f7-4336-ad5e-ef956da82898", "id": 0}]=0 v4) ==== 112+0+2 (crc 0 0 0) 0x7f3e08013b00 con 0x7f3e14114d50 2024-09-22T03:46:44.691 INFO:teuthology.orchestra.run.smithi123.stdout:0 2024-09-22T03:46:44.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.691+0000 7f3e1b0fd640 1 -- 172.21.15.123:0/896530139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e14114d50 msgr2=0x7f3e141a98b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:44.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.691+0000 7f3e1b0fd640 1 --2- 172.21.15.123:0/896530139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e14114d50 0x7f3e141a98b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:44.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.691+0000 7f3e1b0fd640 1 -- 172.21.15.123:0/896530139 shutdown_connections 2024-09-22T03:46:44.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.691+0000 7f3e1b0fd640 1 --2- 172.21.15.123:0/896530139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e14114d50 0x7f3e141a98b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:44.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.691+0000 7f3e1b0fd640 1 -- 172.21.15.123:0/896530139 >> 172.21.15.123:0/896530139 conn(0x7f3e1406fab0 msgr2=0x7f3e14113d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:44.692 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.691+0000 7f3e1b0fd640 1 -- 172.21.15.123:0/896530139 shutdown_connections 2024-09-22T03:46:44.693 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.691+0000 7f3e1b0fd640 1 -- 172.21.15.123:0/896530139 wait complete. 2024-09-22T03:46:44.702 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new d04614be-cd17-47ac-a9ec-7708527db672 1 2024-09-22T03:46:44.809 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.802+0000 7fa9b3e46640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:44.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.808+0000 7fa9b3e46640 1 Processor -- start 2024-09-22T03:46:44.810 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.809+0000 7fa9b3e46640 1 -- start start 2024-09-22T03:46:44.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.809+0000 7fa9b3e46640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9ac1199a0 0x7fa9ac119db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:44.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.809+0000 7fa9b3e46640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9ac115870 con 0x7fa9ac114d50 2024-09-22T03:46:44.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.809+0000 7fa9b3e46640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa9ac115a00 con 0x7fa9ac1199a0 2024-09-22T03:46:44.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.809+0000 7fa9b1be2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9ac1199a0 0x7fa9ac119db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:44.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.809+0000 7fa9b13e1640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa9ac114d50 0x7fa9ac115160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.123:59966/0 (socket says 172.21.15.123:59966) 2024-09-22T03:46:44.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.809+0000 7fa9b1be2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9ac1199a0 0x7fa9ac119db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.123:52024/0 (socket says 172.21.15.123:52024) 2024-09-22T03:46:44.811 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.809+0000 7fa9b13e1640 1 -- 172.21.15.123:0/3924123912 learned_addr learned my addr 172.21.15.123:0/3924123912 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-09-22T03:46:44.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.810+0000 7fa9b1be2640 1 -- 172.21.15.123:0/3924123912 >> v1:172.21.15.3:6789/0 conn(0x7fa9ac114d50 legacy=0x7fa9ac115160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:44.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.810+0000 7fa9b1be2640 1 -- 172.21.15.123:0/3924123912 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ac115b70 con 0x7fa9ac1199a0 2024-09-22T03:46:44.812 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.810+0000 7fa9b1be2640 1 --2- 172.21.15.123:0/3924123912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac1199a0 0x7fa9ac119db0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e0e22c34bfa0bf server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.810+0000 7fa9b0be0640 1 -- 172.21.15.123:0/3924123912 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa9a8009d90 con 0x7fa9ac1199a0 2024-09-22T03:46:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.810+0000 7fa9b0be0640 1 -- 172.21.15.123:0/3924123912 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa9a800e230 con 0x7fa9ac1199a0 2024-09-22T03:46:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.810+0000 7fa9b0be0640 1 -- 172.21.15.123:0/3924123912 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa9a800e4d0 con 0x7fa9ac1199a0 2024-09-22T03:46:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.811+0000 7fa9b3e46640 1 -- 172.21.15.123:0/3924123912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac1199a0 msgr2=0x7fa9ac119db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.811+0000 7fa9b3e46640 1 --2- 172.21.15.123:0/3924123912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac1199a0 0x7fa9ac119db0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.811+0000 7fa9b3e46640 1 -- 172.21.15.123:0/3924123912 shutdown_connections 2024-09-22T03:46:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.811+0000 7fa9b3e46640 1 --2- 172.21.15.123:0/3924123912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac1199a0 0x7fa9ac119db0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.811+0000 7fa9b3e46640 1 -- 172.21.15.123:0/3924123912 >> 172.21.15.123:0/3924123912 conn(0x7fa9ac06fab0 msgr2=0x7fa9ac06fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.811+0000 7fa9b3e46640 1 -- 172.21.15.123:0/3924123912 shutdown_connections 2024-09-22T03:46:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.811+0000 7fa9b3e46640 1 -- 172.21.15.123:0/3924123912 wait complete. 2024-09-22T03:46:44.813 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.811+0000 7fa9b3e46640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.812+0000 7fa9b3e46640 1 Processor -- start 2024-09-22T03:46:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.812+0000 7fa9b3e46640 1 -- start start 2024-09-22T03:46:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.812+0000 7fa9b3e46640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac114d50 0x7fa9ac1a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.812+0000 7fa9b3e46640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac1a9e80 con 0x7fa9ac114d50 2024-09-22T03:46:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.813+0000 7fa9b1be2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac114d50 0x7fa9ac1a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.813+0000 7fa9b1be2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac114d50 0x7fa9ac1a98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.123:52028/0 (socket says 172.21.15.123:52028) 2024-09-22T03:46:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.813+0000 7fa9b1be2640 1 -- 172.21.15.123:0/3276876195 learned_addr learned my addr 172.21.15.123:0/3276876195 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-22T03:46:44.815 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.813+0000 7fa9b1be2640 1 -- 172.21.15.123:0/3276876195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ac1a9fc0 con 0x7fa9ac114d50 2024-09-22T03:46:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.813+0000 7fa9b1be2640 1 --2- 172.21.15.123:0/3276876195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac114d50 0x7fa9ac1a98b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.813+0000 7fa9a27fc640 1 -- 172.21.15.123:0/3276876195 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa9a8014620 con 0x7fa9ac114d50 2024-09-22T03:46:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.813+0000 7fa9a27fc640 1 -- 172.21.15.123:0/3276876195 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa9a8014780 con 0x7fa9ac114d50 2024-09-22T03:46:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.813+0000 7fa9b3e46640 1 -- 172.21.15.123:0/3276876195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ac1aa210 con 0x7fa9ac114d50 2024-09-22T03:46:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.813+0000 7fa9a27fc640 1 -- 172.21.15.123:0/3276876195 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa9a8014a20 con 0x7fa9ac114d50 2024-09-22T03:46:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.813+0000 7fa9b3e46640 1 -- 172.21.15.123:0/3276876195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ac1af110 con 0x7fa9ac114d50 2024-09-22T03:46:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.814+0000 7fa9a27fc640 1 -- 172.21.15.123:0/3276876195 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7fa9a8013400 con 0x7fa9ac114d50 2024-09-22T03:46:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.814+0000 7fa9a27fc640 1 -- 172.21.15.123:0/3276876195 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 862+0+0 (crc 0 0 0) 0x7fa9a8019040 con 0x7fa9ac114d50 2024-09-22T03:46:44.816 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.814+0000 7fa9b3e46640 1 -- 172.21.15.123:0/3276876195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9ac120ba0 con 0x7fa9ac114d50 2024-09-22T03:46:44.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.817+0000 7fa9a27fc640 1 -- 172.21.15.123:0/3276876195 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (crc 0 0 0) 0x7fa9a8013840 con 0x7fa9ac114d50 2024-09-22T03:46:44.889 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.887+0000 7fa9b3e46640 1 -- 172.21.15.123:0/3276876195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d04614be-cd17-47ac-a9ec-7708527db672", "id": 1} v 0) -- 0x7fa9ac120d80 con 0x7fa9ac114d50 2024-09-22T03:46:44.890 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.889+0000 7fa9a27fc640 1 -- 172.21.15.123:0/3276876195 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d04614be-cd17-47ac-a9ec-7708527db672", "id": 1}]=0 v5) ==== 112+0+2 (crc 0 0 0) 0x7fa9a8013b30 con 0x7fa9ac114d50 2024-09-22T03:46:44.890 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-09-22T03:46:44.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.890+0000 7fa9b3e46640 1 -- 172.21.15.123:0/3276876195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac114d50 msgr2=0x7fa9ac1a98b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:44.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.890+0000 7fa9b3e46640 1 --2- 172.21.15.123:0/3276876195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac114d50 0x7fa9ac1a98b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:44.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.890+0000 7fa9b3e46640 1 -- 172.21.15.123:0/3276876195 shutdown_connections 2024-09-22T03:46:44.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.890+0000 7fa9b3e46640 1 --2- 172.21.15.123:0/3276876195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ac114d50 0x7fa9ac1a98b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:44.891 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.890+0000 7fa9b3e46640 1 -- 172.21.15.123:0/3276876195 >> 172.21.15.123:0/3276876195 conn(0x7fa9ac06fab0 msgr2=0x7fa9ac113d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:44.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.890+0000 7fa9b3e46640 1 -- 172.21.15.123:0/3276876195 shutdown_connections 2024-09-22T03:46:44.892 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:44.890+0000 7fa9b3e46640 1 -- 172.21.15.123:0/3276876195 wait complete. 2024-09-22T03:46:44.902 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 8cf57c0d-5e18-448c-a419-e2d417a2d59b 2 2024-09-22T03:46:45.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.001+0000 7f115b911640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:45.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.007+0000 7f115b911640 1 Processor -- start 2024-09-22T03:46:45.008 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.007+0000 7f115b911640 1 -- start start 2024-09-22T03:46:45.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.008+0000 7f115b911640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f115411d8e0 0x7f115411fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:45.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.008+0000 7f115b911640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11541202d0 con 0x7f11541199a0 2024-09-22T03:46:45.009 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.008+0000 7f115b911640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1154121490 con 0x7f115411d8e0 2024-09-22T03:46:45.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.008+0000 7f1158eac640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f115411d8e0 0x7f115411fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:45.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.008+0000 7f11596ad640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f11541199a0 0x7f1154119db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.123:59982/0 (socket says 172.21.15.123:59982) 2024-09-22T03:46:45.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.008+0000 7f1158eac640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f115411d8e0 0x7f115411fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.123:52032/0 (socket says 172.21.15.123:52032) 2024-09-22T03:46:45.010 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.008+0000 7f11596ad640 1 -- 172.21.15.123:0/1214493944 learned_addr learned my addr 172.21.15.123:0/1214493944 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-09-22T03:46:45.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.008+0000 7f114bfff640 1 -- 172.21.15.123:0/1214493944 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1505388674 0 0) 0x7f11541202d0 con 0x7f11541199a0 2024-09-22T03:46:45.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.008+0000 7f114bfff640 1 -- 172.21.15.123:0/1214493944 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1140003580 con 0x7f11541199a0 2024-09-22T03:46:45.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.009+0000 7f1158eac640 1 -- 172.21.15.123:0/1214493944 >> v1:172.21.15.3:6789/0 conn(0x7f11541199a0 legacy=0x7f1154119db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:45.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.009+0000 7f1158eac640 1 -- 172.21.15.123:0/1214493944 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1154121600 con 0x7f115411d8e0 2024-09-22T03:46:45.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.009+0000 7f1158eac640 1 --2- 172.21.15.123:0/1214493944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f115411d8e0 0x7f115411fd00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cda85caa9cb780 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:45.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.009+0000 7f114bfff640 1 -- 172.21.15.123:0/1214493944 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1150009d90 con 0x7f115411d8e0 2024-09-22T03:46:45.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.009+0000 7f114bfff640 1 -- 172.21.15.123:0/1214493944 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f115000e230 con 0x7f115411d8e0 2024-09-22T03:46:45.011 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.009+0000 7f114bfff640 1 -- 172.21.15.123:0/1214493944 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f115000e4d0 con 0x7f115411d8e0 2024-09-22T03:46:45.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.010+0000 7f115b911640 1 -- 172.21.15.123:0/1214493944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f115411d8e0 msgr2=0x7f115411fd00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:45.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.010+0000 7f115b911640 1 --2- 172.21.15.123:0/1214493944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f115411d8e0 0x7f115411fd00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:45.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.010+0000 7f115b911640 1 -- 172.21.15.123:0/1214493944 shutdown_connections 2024-09-22T03:46:45.012 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.010+0000 7f115b911640 1 --2- 172.21.15.123:0/1214493944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f115411d8e0 0x7f115411fd00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:45.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.010+0000 7f115b911640 1 -- 172.21.15.123:0/1214493944 >> 172.21.15.123:0/1214493944 conn(0x7f115406fab0 msgr2=0x7f115406fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:45.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.010+0000 7f115b911640 1 -- 172.21.15.123:0/1214493944 shutdown_connections 2024-09-22T03:46:45.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.010+0000 7f115b911640 1 -- 172.21.15.123:0/1214493944 wait complete. 2024-09-22T03:46:45.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.010+0000 7f115b911640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:45.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.011+0000 7f115b911640 1 Processor -- start 2024-09-22T03:46:45.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.011+0000 7f115b911640 1 -- start start 2024-09-22T03:46:45.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.011+0000 7f115b911640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11541199a0 0x7f11541a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:45.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.011+0000 7f115b911640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f11541a9cd0 con 0x7f11541199a0 2024-09-22T03:46:45.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.012+0000 7f11596ad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11541199a0 0x7f11541a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:45.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.012+0000 7f11596ad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11541199a0 0x7f11541a9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.123:52040/0 (socket says 172.21.15.123:52040) 2024-09-22T03:46:45.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.012+0000 7f11596ad640 1 -- 172.21.15.123:0/749098125 learned_addr learned my addr 172.21.15.123:0/749098125 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-22T03:46:45.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.012+0000 7f11596ad640 1 -- 172.21.15.123:0/749098125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11541a9e10 con 0x7f11541199a0 2024-09-22T03:46:45.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.012+0000 7f11596ad640 1 --2- 172.21.15.123:0/749098125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11541199a0 0x7f11541a9700 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:45.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.012+0000 7f1149ffb640 1 -- 172.21.15.123:0/749098125 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1144005a60 con 0x7f11541199a0 2024-09-22T03:46:45.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.012+0000 7f1149ffb640 1 -- 172.21.15.123:0/749098125 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f1144005bc0 con 0x7f11541199a0 2024-09-22T03:46:45.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.012+0000 7f115b911640 1 -- 172.21.15.123:0/749098125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11541aa0c0 con 0x7f11541199a0 2024-09-22T03:46:45.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.012+0000 7f1149ffb640 1 -- 172.21.15.123:0/749098125 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1144005e60 con 0x7f11541199a0 2024-09-22T03:46:45.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.013+0000 7f1149ffb640 1 -- 172.21.15.123:0/749098125 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f114400cb80 con 0x7f11541199a0 2024-09-22T03:46:45.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.013+0000 7f115b911640 1 -- 172.21.15.123:0/749098125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11541aef60 con 0x7f11541199a0 2024-09-22T03:46:45.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.013+0000 7f1149ffb640 1 -- 172.21.15.123:0/749098125 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 977+0+0 (crc 0 0 0) 0x7f114400e1c0 con 0x7f11541199a0 2024-09-22T03:46:45.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.014+0000 7f115b911640 1 -- 172.21.15.123:0/749098125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1154120f20 con 0x7f11541199a0 2024-09-22T03:46:45.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.017+0000 7f1149ffb640 1 -- 172.21.15.123:0/749098125 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (crc 0 0 0) 0x7f11440070b0 con 0x7f11541199a0 2024-09-22T03:46:45.088 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.087+0000 7f115b911640 1 -- 172.21.15.123:0/749098125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8cf57c0d-5e18-448c-a419-e2d417a2d59b", "id": 2} v 0) -- 0x7f11541207c0 con 0x7f11541199a0 2024-09-22T03:46:45.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.092+0000 7f1149ffb640 1 -- 172.21.15.123:0/749098125 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8cf57c0d-5e18-448c-a419-e2d417a2d59b", "id": 2}]=0 v6) ==== 112+0+2 (crc 0 0 0) 0x7f1144007320 con 0x7f11541199a0 2024-09-22T03:46:45.093 INFO:teuthology.orchestra.run.smithi123.stdout:2 2024-09-22T03:46:45.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.094+0000 7f115b911640 1 -- 172.21.15.123:0/749098125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11541199a0 msgr2=0x7f11541a9700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:45.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.094+0000 7f115b911640 1 --2- 172.21.15.123:0/749098125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11541199a0 0x7f11541a9700 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:45.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.094+0000 7f115b911640 1 -- 172.21.15.123:0/749098125 shutdown_connections 2024-09-22T03:46:45.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.094+0000 7f115b911640 1 --2- 172.21.15.123:0/749098125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11541199a0 0x7f11541a9700 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:45.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.094+0000 7f115b911640 1 -- 172.21.15.123:0/749098125 >> 172.21.15.123:0/749098125 conn(0x7f115406fab0 msgr2=0x7f115411b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:45.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.094+0000 7f115b911640 1 -- 172.21.15.123:0/749098125 shutdown_connections 2024-09-22T03:46:45.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.094+0000 7f115b911640 1 -- 172.21.15.123:0/749098125 wait complete. 2024-09-22T03:46:45.109 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 15079b2b-ea92-49f2-8ab7-eb6edd581ce5 3 2024-09-22T03:46:45.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.207+0000 7efe962fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:45.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.213+0000 7efe962fd640 1 Processor -- start 2024-09-22T03:46:45.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.213+0000 7efe962fd640 1 -- start start 2024-09-22T03:46:45.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.214+0000 7efe962fd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efe9011d8e0 0x7efe9011fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:45.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.214+0000 7efe962fd640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe901202d0 con 0x7efe901199a0 2024-09-22T03:46:45.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.214+0000 7efe962fd640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7efe90121490 con 0x7efe9011d8e0 2024-09-22T03:46:45.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.214+0000 7efe8f7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efe9011d8e0 0x7efe9011fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:45.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.214+0000 7efe8ffff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7efe901199a0 0x7efe90119db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.123:59984/0 (socket says 172.21.15.123:59984) 2024-09-22T03:46:45.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.214+0000 7efe8ffff640 1 -- 172.21.15.123:0/1448187837 learned_addr learned my addr 172.21.15.123:0/1448187837 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-09-22T03:46:45.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.214+0000 7efe8f7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efe9011d8e0 0x7efe9011fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.123:52048/0 (socket says 172.21.15.123:52048) 2024-09-22T03:46:45.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.214+0000 7efe8effd640 1 -- 172.21.15.123:0/1448187837 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1909297025 0 0) 0x7efe901202d0 con 0x7efe901199a0 2024-09-22T03:46:45.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.214+0000 7efe8effd640 1 -- 172.21.15.123:0/1448187837 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe7c003580 con 0x7efe901199a0 2024-09-22T03:46:45.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.215+0000 7efe8f7fe640 1 -- 172.21.15.123:0/1448187837 >> v1:172.21.15.3:6789/0 conn(0x7efe901199a0 legacy=0x7efe90119db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:45.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.215+0000 7efe8f7fe640 1 -- 172.21.15.123:0/1448187837 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe90121600 con 0x7efe9011d8e0 2024-09-22T03:46:45.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.215+0000 7efe8f7fe640 1 --2- 172.21.15.123:0/1448187837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe9011d8e0 0x7efe9011fd00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca5fb29b7848af6 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:45.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.215+0000 7efe8effd640 1 -- 172.21.15.123:0/1448187837 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efe78009d90 con 0x7efe9011d8e0 2024-09-22T03:46:45.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.215+0000 7efe8effd640 1 -- 172.21.15.123:0/1448187837 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7efe7800e230 con 0x7efe9011d8e0 2024-09-22T03:46:45.217 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.215+0000 7efe8effd640 1 -- 172.21.15.123:0/1448187837 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efe7800e4d0 con 0x7efe9011d8e0 2024-09-22T03:46:45.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.216+0000 7efe962fd640 1 -- 172.21.15.123:0/1448187837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe9011d8e0 msgr2=0x7efe9011fd00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:45.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.216+0000 7efe962fd640 1 --2- 172.21.15.123:0/1448187837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe9011d8e0 0x7efe9011fd00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:45.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.216+0000 7efe962fd640 1 -- 172.21.15.123:0/1448187837 shutdown_connections 2024-09-22T03:46:45.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.216+0000 7efe962fd640 1 --2- 172.21.15.123:0/1448187837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe9011d8e0 0x7efe9011fd00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:45.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.216+0000 7efe962fd640 1 -- 172.21.15.123:0/1448187837 >> 172.21.15.123:0/1448187837 conn(0x7efe9006fab0 msgr2=0x7efe9006fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:45.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.216+0000 7efe962fd640 1 -- 172.21.15.123:0/1448187837 shutdown_connections 2024-09-22T03:46:45.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.216+0000 7efe962fd640 1 -- 172.21.15.123:0/1448187837 wait complete. 2024-09-22T03:46:45.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.216+0000 7efe962fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:45.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.217+0000 7efe962fd640 1 Processor -- start 2024-09-22T03:46:45.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.217+0000 7efe962fd640 1 -- start start 2024-09-22T03:46:45.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.217+0000 7efe962fd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe901199a0 0x7efe901a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:45.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.217+0000 7efe962fd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efe901a9cd0 con 0x7efe901199a0 2024-09-22T03:46:45.220 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.218+0000 7efe8ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe901199a0 0x7efe901a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:45.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.218+0000 7efe8ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe901199a0 0x7efe901a9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.123:52062/0 (socket says 172.21.15.123:52062) 2024-09-22T03:46:45.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.218+0000 7efe8ffff640 1 -- 172.21.15.123:0/805017244 learned_addr learned my addr 172.21.15.123:0/805017244 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-22T03:46:45.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.218+0000 7efe8ffff640 1 -- 172.21.15.123:0/805017244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe901a9e10 con 0x7efe901199a0 2024-09-22T03:46:45.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.218+0000 7efe8ffff640 1 --2- 172.21.15.123:0/805017244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe901199a0 0x7efe901a9700 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:45.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.218+0000 7efe8cff9640 1 -- 172.21.15.123:0/805017244 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efe80005a60 con 0x7efe901199a0 2024-09-22T03:46:45.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.219+0000 7efe8cff9640 1 -- 172.21.15.123:0/805017244 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7efe80005bc0 con 0x7efe901199a0 2024-09-22T03:46:45.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.219+0000 7efe962fd640 1 -- 172.21.15.123:0/805017244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe901aa0c0 con 0x7efe901199a0 2024-09-22T03:46:45.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.219+0000 7efe8cff9640 1 -- 172.21.15.123:0/805017244 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efe80005e60 con 0x7efe901199a0 2024-09-22T03:46:45.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.219+0000 7efe962fd640 1 -- 172.21.15.123:0/805017244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe901aef60 con 0x7efe901199a0 2024-09-22T03:46:45.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.219+0000 7efe8cff9640 1 -- 172.21.15.123:0/805017244 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7efe8000cb80 con 0x7efe901199a0 2024-09-22T03:46:45.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.219+0000 7efe8cff9640 1 -- 172.21.15.123:0/805017244 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1092+0+0 (crc 0 0 0) 0x7efe8000fb40 con 0x7efe901199a0 2024-09-22T03:46:45.221 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.219+0000 7efe962fd640 1 -- 172.21.15.123:0/805017244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe90120f20 con 0x7efe901199a0 2024-09-22T03:46:45.223 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.222+0000 7efe8cff9640 1 -- 172.21.15.123:0/805017244 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (crc 0 0 0) 0x7efe8000f230 con 0x7efe901199a0 2024-09-22T03:46:45.294 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.292+0000 7efe962fd640 1 -- 172.21.15.123:0/805017244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd new", "uuid": "15079b2b-ea92-49f2-8ab7-eb6edd581ce5", "id": 3} v 0) -- 0x7efe9011d8e0 con 0x7efe901199a0 2024-09-22T03:46:45.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.298+0000 7efe8cff9640 1 -- 172.21.15.123:0/805017244 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "15079b2b-ea92-49f2-8ab7-eb6edd581ce5", "id": 3}]=0 v7) ==== 112+0+2 (crc 0 0 0) 0x7efe8000fe70 con 0x7efe901199a0 2024-09-22T03:46:45.299 INFO:teuthology.orchestra.run.smithi123.stdout:3 2024-09-22T03:46:45.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.299+0000 7efe962fd640 1 -- 172.21.15.123:0/805017244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe901199a0 msgr2=0x7efe901a9700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:45.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.299+0000 7efe962fd640 1 --2- 172.21.15.123:0/805017244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe901199a0 0x7efe901a9700 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:45.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.299+0000 7efe962fd640 1 -- 172.21.15.123:0/805017244 shutdown_connections 2024-09-22T03:46:45.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.299+0000 7efe962fd640 1 --2- 172.21.15.123:0/805017244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe901199a0 0x7efe901a9700 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:45.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.299+0000 7efe962fd640 1 -- 172.21.15.123:0/805017244 >> 172.21.15.123:0/805017244 conn(0x7efe9006fab0 msgr2=0x7efe9011b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:45.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.299+0000 7efe962fd640 1 -- 172.21.15.123:0/805017244 shutdown_connections 2024-09-22T03:46:45.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-22T03:46:45.299+0000 7efe962fd640 1 -- 172.21.15.123:0/805017244 wait complete. 2024-09-22T03:46:45.311 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2024-09-22T03:46:45.348 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-22T03:46:45.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-09-22T03:46:45.391 INFO:tasks.ceph.osd.0:Started 2024-09-22T03:46:45.391 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2024-09-22T03:46:45.424 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-22T03:46:45.424 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-22T03:46:45.466 INFO:tasks.ceph.osd.1:Started 2024-09-22T03:46:45.466 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2024-09-22T03:46:45.496 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-22T03:46:45.496 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-09-22T03:46:45.538 INFO:tasks.ceph.osd.2:Started 2024-09-22T03:46:45.538 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2024-09-22T03:46:45.568 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-22T03:46:45.568 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-09-22T03:46:45.610 INFO:tasks.ceph.osd.3:Started 2024-09-22T03:46:45.611 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T03:46:45.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.725+0000 7f9cad591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:45.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.730+0000 7f9cad591640 1 Processor -- start 2024-09-22T03:46:45.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.730+0000 7f9cad591640 1 -- start start 2024-09-22T03:46:45.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.732+0000 7f9cad591640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ca80a4930 0x7f9ca813b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:45.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.732+0000 7f9cad591640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca806ec50 con 0x7f9ca80a3e10 2024-09-22T03:46:45.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.732+0000 7f9cad591640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f9ca80a6130 con 0x7f9ca80a4930 2024-09-22T03:46:45.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.732+0000 7f9ca67fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9ca80a4930 0x7f9ca813b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:45.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.732+0000 7f9ca6ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9ca80a3e10 0x7f9ca80a4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45658/0 (socket says 172.21.15.3:45658) 2024-09-22T03:46:45.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.732+0000 7f9ca6ffd640 1 -- 172.21.15.3:0/4137356946 learned_addr learned my addr 172.21.15.3:0/4137356946 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T03:46:45.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.732+0000 7f9ca67fc640 1 -- 172.21.15.3:0/4137356946 >> v1:172.21.15.3:6789/0 conn(0x7f9ca80a3e10 legacy=0x7f9ca80a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:45.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.732+0000 7f9ca67fc640 1 -- 172.21.15.3:0/4137356946 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca80a5110 con 0x7f9ca80a4930 2024-09-22T03:46:45.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.732+0000 7f9ca5ffb640 1 -- 172.21.15.3:0/4137356946 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3647966621 0 0) 0x7f9ca806ec50 con 0x7f9ca80a3e10 2024-09-22T03:46:45.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.733+0000 7f9ca67fc640 1 --2- 172.21.15.3:0/4137356946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ca80a4930 0x7f9ca813b2d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b19978f0e0a780 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:45.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.733+0000 7f9ca5ffb640 1 -- 172.21.15.3:0/4137356946 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9c9c009d90 con 0x7f9ca80a4930 2024-09-22T03:46:45.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.733+0000 7f9ca5ffb640 1 -- 172.21.15.3:0/4137356946 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f9c9c00e230 con 0x7f9ca80a4930 2024-09-22T03:46:45.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.733+0000 7f9ca5ffb640 1 -- 172.21.15.3:0/4137356946 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9c9c00e4d0 con 0x7f9ca80a4930 2024-09-22T03:46:45.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.734+0000 7f9cad591640 1 -- 172.21.15.3:0/4137356946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ca80a4930 msgr2=0x7f9ca813b2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:45.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.734+0000 7f9cad591640 1 --2- 172.21.15.3:0/4137356946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ca80a4930 0x7f9ca813b2d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:45.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.734+0000 7f9cad591640 1 -- 172.21.15.3:0/4137356946 shutdown_connections 2024-09-22T03:46:45.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.734+0000 7f9cad591640 1 --2- 172.21.15.3:0/4137356946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ca80a4930 0x7f9ca813b2d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:45.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.734+0000 7f9cad591640 1 -- 172.21.15.3:0/4137356946 >> 172.21.15.3:0/4137356946 conn(0x7f9ca80a74f0 msgr2=0x7f9ca80a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:45.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.734+0000 7f9cad591640 1 -- 172.21.15.3:0/4137356946 shutdown_connections 2024-09-22T03:46:45.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.734+0000 7f9cad591640 1 -- 172.21.15.3:0/4137356946 wait complete. 2024-09-22T03:46:45.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.734+0000 7f9cad591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:45.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.736+0000 7f9cad591640 1 Processor -- start 2024-09-22T03:46:45.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.736+0000 7f9cad591640 1 -- start start 2024-09-22T03:46:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.737+0000 7f9cad591640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ca80a3e10 0x7f9ca8124230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.737+0000 7f9cad591640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca8125b10 con 0x7f9ca80a3e10 2024-09-22T03:46:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.738+0000 7f9ca6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ca80a3e10 0x7f9ca8124230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.738+0000 7f9ca6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ca80a3e10 0x7f9ca8124230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46430/0 (socket says 172.21.15.3:46430) 2024-09-22T03:46:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.738+0000 7f9ca6ffd640 1 -- 172.21.15.3:0/4190541019 learned_addr learned my addr 172.21.15.3:0/4190541019 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.738+0000 7f9ca6ffd640 1 -- 172.21.15.3:0/4190541019 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca8123de0 con 0x7f9ca80a3e10 2024-09-22T03:46:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.738+0000 7f9ca6ffd640 1 --2- 172.21.15.3:0/4190541019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ca80a3e10 0x7f9ca8124230 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.738+0000 7f9c8f7fe640 1 -- 172.21.15.3:0/4190541019 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9c98004b80 con 0x7f9ca80a3e10 2024-09-22T03:46:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.738+0000 7f9c8f7fe640 1 -- 172.21.15.3:0/4190541019 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f9c98004ce0 con 0x7f9ca80a3e10 2024-09-22T03:46:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.738+0000 7f9cad591640 1 -- 172.21.15.3:0/4190541019 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca81252d0 con 0x7f9ca80a3e10 2024-09-22T03:46:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.738+0000 7f9c8f7fe640 1 -- 172.21.15.3:0/4190541019 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9c98007120 con 0x7f9ca80a3e10 2024-09-22T03:46:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.738+0000 7f9cad591640 1 -- 172.21.15.3:0/4190541019 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca8127750 con 0x7f9ca80a3e10 2024-09-22T03:46:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.739+0000 7f9c8f7fe640 1 -- 172.21.15.3:0/4190541019 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f9c9800c520 con 0x7f9ca80a3e10 2024-09-22T03:46:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.739+0000 7f9c8f7fe640 1 -- 172.21.15.3:0/4190541019 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1207+0+0 (crc 0 0 0) 0x7f9c98007660 con 0x7f9ca80a3e10 2024-09-22T03:46:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.740+0000 7f9cad591640 1 -- 172.21.15.3:0/4190541019 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca80a56d0 con 0x7f9ca80a3e10 2024-09-22T03:46:45.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.742+0000 7f9c8f7fe640 1 -- 172.21.15.3:0/4190541019 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (crc 0 0 0) 0x7f9c9800f230 con 0x7f9ca80a3e10 2024-09-22T03:46:45.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.804+0000 7f9cad591640 1 -- 172.21.15.3:0/4190541019 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ca8077360 con 0x7f9ca80a3e10 2024-09-22T03:46:45.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.804+0000 7f9c8f7fe640 1 -- 172.21.15.3:0/4190541019 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) ==== 74+0+4152 (crc 0 0 0) 0x7f9c9800c880 con 0x7f9ca80a3e10 2024-09-22T03:46:45.806 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:45.806 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"fsid":"012c124d-1263-4b17-8b29-e14858734658","created":"2024-09-22T03:46:43.165321+0000","modified":"2024-09-22T03:46:45.294435+0000","last_up_change":"0.000000","last_in_change":"2024-09-22T03:46:45.294435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"ae27f46d-f8f7-4336-ad5e-ef956da82898","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d04614be-cd17-47ac-a9ec-7708527db672","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8cf57c0d-5e18-448c-a419-e2d417a2d59b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"15079b2b-ea92-49f2-8ab7-eb6edd581ce5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:46:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.805+0000 7f9cad591640 1 -- 172.21.15.3:0/4190541019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ca80a3e10 msgr2=0x7f9ca8124230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.805+0000 7f9cad591640 1 --2- 172.21.15.3:0/4190541019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ca80a3e10 0x7f9ca8124230 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.806+0000 7f9cad591640 1 -- 172.21.15.3:0/4190541019 shutdown_connections 2024-09-22T03:46:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.806+0000 7f9cad591640 1 --2- 172.21.15.3:0/4190541019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ca80a3e10 0x7f9ca8124230 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.806+0000 7f9cad591640 1 -- 172.21.15.3:0/4190541019 >> 172.21.15.3:0/4190541019 conn(0x7f9ca80a74f0 msgr2=0x7f9ca8071390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.806+0000 7f9cad591640 1 -- 172.21.15.3:0/4190541019 shutdown_connections 2024-09-22T03:46:45.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:45.806+0000 7f9cad591640 1 -- 172.21.15.3:0/4190541019 wait complete. 2024-09-22T03:46:45.818 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-22T03:46:45.818 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-22T03:46:46.220 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-22T03:46:46.328 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.322+0000 7f2e9d751640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:46.329 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.328+0000 7f2e9d751640 1 Processor -- start 2024-09-22T03:46:46.329 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.328+0000 7f2e9d751640 1 -- start start 2024-09-22T03:46:46.331 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.329+0000 7f2e9d751640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e980a4930 0x7f2e9813b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:46.331 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.329+0000 7f2e9d751640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e9806ed80 con 0x7f2e980a3e10 2024-09-22T03:46:46.332 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.329+0000 7f2e9d751640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2e980a6380 con 0x7f2e980a4930 2024-09-22T03:46:46.332 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.329+0000 7f2e967fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2e980a4930 0x7f2e9813b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:46.332 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.329+0000 7f2e96ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2e980a3e10 0x7f2e980a4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45660/0 (socket says 172.21.15.3:45660) 2024-09-22T03:46:46.332 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.329+0000 7f2e96ffd640 1 -- 172.21.15.3:0/3115169998 learned_addr learned my addr 172.21.15.3:0/3115169998 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T03:46:46.332 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.330+0000 7f2e967fc640 1 -- 172.21.15.3:0/3115169998 >> v1:172.21.15.3:6789/0 conn(0x7f2e980a3e10 legacy=0x7f2e980a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:46.332 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.330+0000 7f2e967fc640 1 -- 172.21.15.3:0/3115169998 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9806f7a0 con 0x7f2e980a4930 2024-09-22T03:46:46.332 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.330+0000 7f2e95ffb640 1 -- 172.21.15.3:0/3115169998 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4023153302 0 0) 0x7f2e9806ed80 con 0x7f2e980a3e10 2024-09-22T03:46:46.332 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.330+0000 7f2e967fc640 1 --2- 172.21.15.3:0/3115169998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e980a4930 0x7f2e9813b4e0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209a7f7cfbe6343a server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:46.332 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.330+0000 7f2e95ffb640 1 -- 172.21.15.3:0/3115169998 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2e80009d90 con 0x7f2e980a4930 2024-09-22T03:46:46.332 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.330+0000 7f2e95ffb640 1 -- 172.21.15.3:0/3115169998 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2e8000e230 con 0x7f2e980a4930 2024-09-22T03:46:46.332 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.330+0000 7f2e95ffb640 1 -- 172.21.15.3:0/3115169998 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2e8000e4d0 con 0x7f2e980a4930 2024-09-22T03:46:46.333 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.331+0000 7f2e9d751640 1 -- 172.21.15.3:0/3115169998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e980a4930 msgr2=0x7f2e9813b4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:46.333 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.331+0000 7f2e9d751640 1 --2- 172.21.15.3:0/3115169998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e980a4930 0x7f2e9813b4e0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:46.333 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.331+0000 7f2e9d751640 1 -- 172.21.15.3:0/3115169998 shutdown_connections 2024-09-22T03:46:46.333 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.331+0000 7f2e9d751640 1 --2- 172.21.15.3:0/3115169998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e980a4930 0x7f2e9813b4e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:46.333 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.331+0000 7f2e9d751640 1 -- 172.21.15.3:0/3115169998 >> 172.21.15.3:0/3115169998 conn(0x7f2e980a7700 msgr2=0x7f2e980a7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:46.333 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.331+0000 7f2e9d751640 1 -- 172.21.15.3:0/3115169998 shutdown_connections 2024-09-22T03:46:46.334 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.331+0000 7f2e9d751640 1 -- 172.21.15.3:0/3115169998 wait complete. 2024-09-22T03:46:46.334 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.331+0000 7f2e9d751640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:46.334 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.333+0000 7f2e9d751640 1 Processor -- start 2024-09-22T03:46:46.335 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.333+0000 7f2e9d751640 1 -- start start 2024-09-22T03:46:46.336 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.335+0000 7f2e9d751640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e980a3e10 0x7f2e98126d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:46.337 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.335+0000 7f2e9d751640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e98136f70 con 0x7f2e980a3e10 2024-09-22T03:46:46.337 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.335+0000 7f2e96ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e980a3e10 0x7f2e98126d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:46.337 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.335+0000 7f2e96ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e980a3e10 0x7f2e98126d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46456/0 (socket says 172.21.15.3:46456) 2024-09-22T03:46:46.337 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.335+0000 7f2e96ffd640 1 -- 172.21.15.3:0/372711080 learned_addr learned my addr 172.21.15.3:0/372711080 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:46.337 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.335+0000 7f2e96ffd640 1 -- 172.21.15.3:0/372711080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e98125610 con 0x7f2e980a3e10 2024-09-22T03:46:46.337 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.336+0000 7f2e96ffd640 1 --2- 172.21.15.3:0/372711080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e980a3e10 0x7f2e98126d50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:46.338 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.336+0000 7f2e8f7fe640 1 -- 172.21.15.3:0/372711080 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2e88004b80 con 0x7f2e980a3e10 2024-09-22T03:46:46.338 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.336+0000 7f2e8f7fe640 1 -- 172.21.15.3:0/372711080 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2e88004ce0 con 0x7f2e980a3e10 2024-09-22T03:46:46.338 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.336+0000 7f2e8f7fe640 1 -- 172.21.15.3:0/372711080 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2e88007120 con 0x7f2e980a3e10 2024-09-22T03:46:46.338 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.336+0000 7f2e9d751640 1 -- 172.21.15.3:0/372711080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e98124d60 con 0x7f2e980a3e10 2024-09-22T03:46:46.338 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.336+0000 7f2e9d751640 1 -- 172.21.15.3:0/372711080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e98126430 con 0x7f2e980a3e10 2024-09-22T03:46:46.338 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.336+0000 7f2e8f7fe640 1 -- 172.21.15.3:0/372711080 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (crc 0 0 0) 0x7f2e8800c520 con 0x7f2e980a3e10 2024-09-22T03:46:46.338 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.336+0000 7f2e8f7fe640 1 -- 172.21.15.3:0/372711080 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1207+0+0 (crc 0 0 0) 0x7f2e8800fc00 con 0x7f2e980a3e10 2024-09-22T03:46:46.338 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.337+0000 7f2e9d751640 1 -- 172.21.15.3:0/372711080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e981be0c0 con 0x7f2e980a3e10 2024-09-22T03:46:46.340 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.339+0000 7f2e8f7fe640 1 -- 172.21.15.3:0/372711080 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (crc 0 0 0) 0x7f2e981be0c0 con 0x7f2e980a3e10 2024-09-22T03:46:46.403 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.402+0000 7f2e9d751640 1 -- 172.21.15.3:0/372711080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e9806f320 con 0x7f2e980a3e10 2024-09-22T03:46:46.404 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.403+0000 7f2e8f7fe640 1 -- 172.21.15.3:0/372711080 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) ==== 74+0+4152 (crc 0 0 0) 0x7f2e88007660 con 0x7f2e980a3e10 2024-09-22T03:46:46.404 INFO:teuthology.misc.health.smithi003.stdout: 2024-09-22T03:46:46.404 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":7,"fsid":"012c124d-1263-4b17-8b29-e14858734658","created":"2024-09-22T03:46:43.165321+0000","modified":"2024-09-22T03:46:45.294435+0000","last_up_change":"0.000000","last_in_change":"2024-09-22T03:46:45.294435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"ae27f46d-f8f7-4336-ad5e-ef956da82898","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d04614be-cd17-47ac-a9ec-7708527db672","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8cf57c0d-5e18-448c-a419-e2d417a2d59b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"15079b2b-ea92-49f2-8ab7-eb6edd581ce5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:46:46.405 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.404+0000 7f2e9d751640 1 -- 172.21.15.3:0/372711080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e980a3e10 msgr2=0x7f2e98126d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:46.405 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.404+0000 7f2e9d751640 1 --2- 172.21.15.3:0/372711080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e980a3e10 0x7f2e98126d50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:46.405 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.404+0000 7f2e9d751640 1 -- 172.21.15.3:0/372711080 shutdown_connections 2024-09-22T03:46:46.405 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.404+0000 7f2e9d751640 1 --2- 172.21.15.3:0/372711080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e980a3e10 0x7f2e98126d50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:46.405 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.404+0000 7f2e9d751640 1 -- 172.21.15.3:0/372711080 >> 172.21.15.3:0/372711080 conn(0x7f2e980a7700 msgr2=0x7f2e981bc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:46.405 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.404+0000 7f2e9d751640 1 -- 172.21.15.3:0/372711080 shutdown_connections 2024-09-22T03:46:46.406 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:46.404+0000 7f2e9d751640 1 -- 172.21.15.3:0/372711080 wait complete. 2024-09-22T03:46:46.415 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-09-22T03:46:48.563 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-22T03:46:48.562+0000 7f030fe57640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-09-22T03:46:52.818 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-22T03:46:52.925 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.919+0000 7fe230951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:52.926 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.924+0000 7fe230951640 1 Processor -- start 2024-09-22T03:46:52.926 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.924+0000 7fe230951640 1 -- start start 2024-09-22T03:46:52.927 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.926+0000 7fe230951640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe22c1349c0 0x7fe22c139110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:52.928 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.926+0000 7fe230951640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe22c09d580 con 0x7fe22c09ccf0 2024-09-22T03:46:52.928 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.926+0000 7fe230951640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe22c09df80 con 0x7fe22c1349c0 2024-09-22T03:46:52.928 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.926+0000 7fe229d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe22c1349c0 0x7fe22c139110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:52.928 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.926+0000 7fe22a59c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe22c09ccf0 0x7fe22c1342b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49660/0 (socket says 172.21.15.3:49660) 2024-09-22T03:46:52.928 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.926+0000 7fe22a59c640 1 -- 172.21.15.3:0/158802480 learned_addr learned my addr 172.21.15.3:0/158802480 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T03:46:52.929 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.927+0000 7fe229d9b640 1 -- 172.21.15.3:0/158802480 >> v1:172.21.15.3:6789/0 conn(0x7fe22c09ccf0 legacy=0x7fe22c1342b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:52.929 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.927+0000 7fe229d9b640 1 -- 172.21.15.3:0/158802480 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c074950 con 0x7fe22c1349c0 2024-09-22T03:46:52.929 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.927+0000 7fe229d9b640 1 --2- 172.21.15.3:0/158802480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c1349c0 0x7fe22c139110 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f5c25bc380a578 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:52.930 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.927+0000 7fe22959a640 1 -- 172.21.15.3:0/158802480 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe214009d90 con 0x7fe22c1349c0 2024-09-22T03:46:52.930 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.927+0000 7fe22959a640 1 -- 172.21.15.3:0/158802480 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe21400e230 con 0x7fe22c1349c0 2024-09-22T03:46:52.930 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.927+0000 7fe22959a640 1 -- 172.21.15.3:0/158802480 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe21400e4d0 con 0x7fe22c1349c0 2024-09-22T03:46:52.930 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.928+0000 7fe230951640 1 -- 172.21.15.3:0/158802480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c1349c0 msgr2=0x7fe22c139110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:52.930 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.928+0000 7fe230951640 1 --2- 172.21.15.3:0/158802480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c1349c0 0x7fe22c139110 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:52.930 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.928+0000 7fe230951640 1 -- 172.21.15.3:0/158802480 shutdown_connections 2024-09-22T03:46:52.930 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.928+0000 7fe230951640 1 --2- 172.21.15.3:0/158802480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c1349c0 0x7fe22c139110 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:52.930 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.928+0000 7fe230951640 1 -- 172.21.15.3:0/158802480 >> 172.21.15.3:0/158802480 conn(0x7fe22c06e6f0 msgr2=0x7fe22c06eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:52.930 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.928+0000 7fe230951640 1 -- 172.21.15.3:0/158802480 shutdown_connections 2024-09-22T03:46:52.930 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.928+0000 7fe230951640 1 -- 172.21.15.3:0/158802480 wait complete. 2024-09-22T03:46:52.930 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.928+0000 7fe230951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:52.931 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.930+0000 7fe230951640 1 Processor -- start 2024-09-22T03:46:52.931 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.930+0000 7fe230951640 1 -- start start 2024-09-22T03:46:52.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.931+0000 7fe230951640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c09ccf0 0x7fe22c1c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:52.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.931+0000 7fe230951640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c120f30 con 0x7fe22c09ccf0 2024-09-22T03:46:52.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.932+0000 7fe22a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c09ccf0 0x7fe22c1c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:52.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.932+0000 7fe22a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c09ccf0 0x7fe22c1c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32906/0 (socket says 172.21.15.3:32906) 2024-09-22T03:46:52.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.932+0000 7fe22a59c640 1 -- 172.21.15.3:0/3290242139 learned_addr learned my addr 172.21.15.3:0/3290242139 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:52.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.932+0000 7fe22a59c640 1 -- 172.21.15.3:0/3290242139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c11ec00 con 0x7fe22c09ccf0 2024-09-22T03:46:52.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.932+0000 7fe22a59c640 1 --2- 172.21.15.3:0/3290242139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c09ccf0 0x7fe22c1c8900 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:52.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.932+0000 7fe222ffd640 1 -- 172.21.15.3:0/3290242139 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe21c0070b0 con 0x7fe22c09ccf0 2024-09-22T03:46:52.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.932+0000 7fe222ffd640 1 -- 172.21.15.3:0/3290242139 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe21c007210 con 0x7fe22c09ccf0 2024-09-22T03:46:52.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.932+0000 7fe230951640 1 -- 172.21.15.3:0/3290242139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22c120be0 con 0x7fe22c09ccf0 2024-09-22T03:46:52.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.932+0000 7fe222ffd640 1 -- 172.21.15.3:0/3290242139 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe21c0074b0 con 0x7fe22c09ccf0 2024-09-22T03:46:52.935 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.932+0000 7fe230951640 1 -- 172.21.15.3:0/3290242139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22c122820 con 0x7fe22c09ccf0 2024-09-22T03:46:52.935 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.933+0000 7fe230951640 1 -- 172.21.15.3:0/3290242139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe22c1342b0 con 0x7fe22c09ccf0 2024-09-22T03:46:52.935 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.933+0000 7fe222ffd640 1 -- 172.21.15.3:0/3290242139 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 50975+0+0 (crc 0 0 0) 0x7fe21c007660 con 0x7fe22c09ccf0 2024-09-22T03:46:52.935 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.934+0000 7fe222ffd640 1 --2- 172.21.15.3:0/3290242139 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe20c03ea70 0x7fe20c040f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:52.935 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.934+0000 7fe222ffd640 1 -- 172.21.15.3:0/3290242139 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 2308+0+0 (crc 0 0 0) 0x7fe21c04fea0 con 0x7fe22c09ccf0 2024-09-22T03:46:52.936 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.934+0000 7fe229d9b640 1 --2- 172.21.15.3:0/3290242139 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe20c03ea70 0x7fe20c040f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:52.936 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.935+0000 7fe229d9b640 1 --2- 172.21.15.3:0/3290242139 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe20c03ea70 0x7fe20c040f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:52.940 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:52.939+0000 7fe222ffd640 1 -- 172.21.15.3:0/3290242139 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7fe21c0096b0 con 0x7fe22c09ccf0 2024-09-22T03:46:53.131 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:53.129+0000 7fe230951640 1 -- 172.21.15.3:0/3290242139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe22c09d580 con 0x7fe22c09ccf0 2024-09-22T03:46:53.131 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:53.130+0000 7fe222ffd640 1 -- 172.21.15.3:0/3290242139 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+6938 (crc 0 0 0) 0x7fe22c09d580 con 0x7fe22c09ccf0 2024-09-22T03:46:53.131 INFO:teuthology.misc.health.smithi003.stdout: 2024-09-22T03:46:53.132 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":10,"fsid":"012c124d-1263-4b17-8b29-e14858734658","created":"2024-09-22T03:46:43.165321+0000","modified":"2024-09-22T03:46:52.275666+0000","last_up_change":"2024-09-22T03:46:50.268789+0000","last_in_change":"2024-09-22T03:46:45.294435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T03:46:51.582570+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae27f46d-f8f7-4336-ad5e-ef956da82898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6801","nonce":2889521405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2889521405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6805","nonce":2889521405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2889521405}]},"public_addr":"172.21.15.3:6801/2889521405","cluster_addr":"172.21.15.3:6800/2889521405","heartbeat_back_addr":"172.21.15.3:6805/2889521405","heartbeat_front_addr":"172.21.15.3:6804/2889521405","state":["exists","up"]},{"osd":1,"uuid":"d04614be-cd17-47ac-a9ec-7708527db672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2784566484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":2784566484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":2784566484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2784566484}]},"public_addr":"172.21.15.3:6802/2784566484","cluster_addr":"172.21.15.3:6803/2784566484","heartbeat_back_addr":"172.21.15.3:6807/2784566484","heartbeat_front_addr":"172.21.15.3:6806/2784566484","state":["exists","up"]},{"osd":2,"uuid":"8cf57c0d-5e18-448c-a419-e2d417a2d59b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":891966684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":891966684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6811","nonce":891966684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":891966684}]},"public_addr":"172.21.15.123:6813/891966684","cluster_addr":"172.21.15.123:6812/891966684","heartbeat_back_addr":"172.21.15.123:6811/891966684","heartbeat_front_addr":"172.21.15.123:6810/891966684","state":["exists","up"]},{"osd":3,"uuid":"15079b2b-ea92-49f2-8ab7-eb6edd581ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4206675901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6805","nonce":4206675901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":4206675901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4206675901}]},"public_addr":"172.21.15.123:6806/4206675901","cluster_addr":"172.21.15.123:6805/4206675901","heartbeat_back_addr":"172.21.15.123:6807/4206675901","heartbeat_front_addr":"172.21.15.123:6804/4206675901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:46:53.134 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:53.133+0000 7fe230951640 1 -- 172.21.15.3:0/3290242139 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe20c03ea70 msgr2=0x7fe20c040f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:53.134 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:53.133+0000 7fe230951640 1 --2- 172.21.15.3:0/3290242139 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe20c03ea70 0x7fe20c040f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:53.134 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:53.133+0000 7fe230951640 1 -- 172.21.15.3:0/3290242139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c09ccf0 msgr2=0x7fe22c1c8900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:53.135 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:53.133+0000 7fe230951640 1 --2- 172.21.15.3:0/3290242139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c09ccf0 0x7fe22c1c8900 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:53.135 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:53.133+0000 7fe230951640 1 -- 172.21.15.3:0/3290242139 shutdown_connections 2024-09-22T03:46:53.135 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:53.133+0000 7fe230951640 1 --2- 172.21.15.3:0/3290242139 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe20c03ea70 0x7fe20c040f80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:53.135 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:53.133+0000 7fe230951640 1 --2- 172.21.15.3:0/3290242139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe22c09ccf0 0x7fe22c1c8900 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:53.135 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:53.133+0000 7fe230951640 1 -- 172.21.15.3:0/3290242139 >> 172.21.15.3:0/3290242139 conn(0x7fe22c06e6f0 msgr2=0x7fe22c0a1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:53.135 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:53.133+0000 7fe230951640 1 -- 172.21.15.3:0/3290242139 shutdown_connections 2024-09-22T03:46:53.135 INFO:teuthology.misc.health.smithi003.stderr:2024-09-22T03:46:53.133+0000 7fe230951640 1 -- 172.21.15.3:0/3290242139 wait complete. 2024-09-22T03:46:53.143 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-09-22T03:46:53.144 INFO:tasks.ceph:Creating RBD pool 2024-09-22T03:46:53.144 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-22T03:46:53.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.247+0000 7fbd864d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:53.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.256+0000 7fbd864d1640 1 Processor -- start 2024-09-22T03:46:53.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.256+0000 7fbd864d1640 1 -- start start 2024-09-22T03:46:53.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.257+0000 7fbd864d1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbd800a1fc0 0x7fbd80073010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:53.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.257+0000 7fbd864d1640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd800743d0 con 0x7fbd80073550 2024-09-22T03:46:53.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.257+0000 7fbd864d1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbd800740c0 con 0x7fbd800a1fc0 2024-09-22T03:46:53.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.258+0000 7fbd7ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbd800a1fc0 0x7fbd80073010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:53.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.258+0000 7fbd7ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbd800a1fc0 0x7fbd80073010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32916/0 (socket says 172.21.15.3:32916) 2024-09-22T03:46:53.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.258+0000 7fbd7f7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbd80073550 0x7fbd80135d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49676/0 (socket says 172.21.15.3:49676) 2024-09-22T03:46:53.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.258+0000 7fbd7ffff640 1 -- 172.21.15.3:0/2579104471 learned_addr learned my addr 172.21.15.3:0/2579104471 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:53.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.258+0000 7fbd7ffff640 1 -- 172.21.15.3:0/2579104471 >> v1:172.21.15.3:6789/0 conn(0x7fbd80073550 legacy=0x7fbd80135d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:53.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.258+0000 7fbd7effd640 1 -- 172.21.15.3:0/2579104471 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4042130643 0 0) 0x7fbd800743d0 con 0x7fbd80073550 2024-09-22T03:46:53.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.258+0000 7fbd7ffff640 1 -- 172.21.15.3:0/2579104471 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd800a3270 con 0x7fbd800a1fc0 2024-09-22T03:46:53.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.258+0000 7fbd7ffff640 1 --2- 172.21.15.3:0/2579104471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd800a1fc0 0x7fbd80073010 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad30cc0408994e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:53.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.258+0000 7fbd7effd640 1 -- 172.21.15.3:0/2579104471 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbd70009e50 con 0x7fbd800a1fc0 2024-09-22T03:46:53.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.258+0000 7fbd7effd640 1 -- 172.21.15.3:0/2579104471 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbd7000e230 con 0x7fbd800a1fc0 2024-09-22T03:46:53.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.258+0000 7fbd7effd640 1 -- 172.21.15.3:0/2579104471 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbd7000e4d0 con 0x7fbd800a1fc0 2024-09-22T03:46:53.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.259+0000 7fbd864d1640 1 -- 172.21.15.3:0/2579104471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd800a1fc0 msgr2=0x7fbd80073010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:53.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.259+0000 7fbd864d1640 1 --2- 172.21.15.3:0/2579104471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd800a1fc0 0x7fbd80073010 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:53.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.259+0000 7fbd864d1640 1 -- 172.21.15.3:0/2579104471 shutdown_connections 2024-09-22T03:46:53.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.259+0000 7fbd864d1640 1 --2- 172.21.15.3:0/2579104471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd800a1fc0 0x7fbd80073010 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:53.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.259+0000 7fbd864d1640 1 -- 172.21.15.3:0/2579104471 >> 172.21.15.3:0/2579104471 conn(0x7fbd8006f0a0 msgr2=0x7fbd800714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:53.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.259+0000 7fbd864d1640 1 -- 172.21.15.3:0/2579104471 shutdown_connections 2024-09-22T03:46:53.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.259+0000 7fbd864d1640 1 -- 172.21.15.3:0/2579104471 wait complete. 2024-09-22T03:46:53.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.259+0000 7fbd864d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:53.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.261+0000 7fbd864d1640 1 Processor -- start 2024-09-22T03:46:53.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.261+0000 7fbd864d1640 1 -- start start 2024-09-22T03:46:53.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.263+0000 7fbd864d1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd80073550 0x7fbd80123ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:53.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.263+0000 7fbd864d1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbd80125f60 con 0x7fbd80073550 2024-09-22T03:46:53.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.263+0000 7fbd7ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd80073550 0x7fbd80123ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:53.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.263+0000 7fbd7ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd80073550 0x7fbd80123ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32918/0 (socket says 172.21.15.3:32918) 2024-09-22T03:46:53.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.263+0000 7fbd7ffff640 1 -- 172.21.15.3:0/3259428704 learned_addr learned my addr 172.21.15.3:0/3259428704 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:53.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.264+0000 7fbd7ffff640 1 -- 172.21.15.3:0/3259428704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd80126210 con 0x7fbd80073550 2024-09-22T03:46:53.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.264+0000 7fbd7ffff640 1 --2- 172.21.15.3:0/3259428704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd80073550 0x7fbd80123ef0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:53.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.264+0000 7fbd7cff9640 1 -- 172.21.15.3:0/3259428704 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbd7000eda0 con 0x7fbd80073550 2024-09-22T03:46:53.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.264+0000 7fbd7cff9640 1 -- 172.21.15.3:0/3259428704 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbd70015420 con 0x7fbd80073550 2024-09-22T03:46:53.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.264+0000 7fbd864d1640 1 -- 172.21.15.3:0/3259428704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd801276e0 con 0x7fbd80073550 2024-09-22T03:46:53.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.264+0000 7fbd7cff9640 1 -- 172.21.15.3:0/3259428704 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbd700156c0 con 0x7fbd80073550 2024-09-22T03:46:53.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.264+0000 7fbd864d1640 1 -- 172.21.15.3:0/3259428704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd80123b50 con 0x7fbd80073550 2024-09-22T03:46:53.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.265+0000 7fbd7cff9640 1 -- 172.21.15.3:0/3259428704 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 50975+0+0 (crc 0 0 0) 0x7fbd70015870 con 0x7fbd80073550 2024-09-22T03:46:53.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.265+0000 7fbd864d1640 1 -- 172.21.15.3:0/3259428704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd80073010 con 0x7fbd80073550 2024-09-22T03:46:53.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.265+0000 7fbd7cff9640 1 --2- 172.21.15.3:0/3259428704 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbd6c03ea70 0x7fbd6c040f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:53.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.265+0000 7fbd7cff9640 1 -- 172.21.15.3:0/3259428704 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 2308+0+0 (crc 0 0 0) 0x7fbd70050d30 con 0x7fbd80073550 2024-09-22T03:46:53.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.266+0000 7fbd7f7fe640 1 --2- 172.21.15.3:0/3259428704 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbd6c03ea70 0x7fbd6c040f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:53.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.266+0000 7fbd7f7fe640 1 --2- 172.21.15.3:0/3259428704 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbd6c03ea70 0x7fbd6c040f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:53.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.271+0000 7fbd7cff9640 1 -- 172.21.15.3:0/3259428704 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7fbd7000f2a0 con 0x7fbd80073550 2024-09-22T03:46:53.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:53.506+0000 7fbd864d1640 1 -- 172.21.15.3:0/3259428704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7fbd80077360 con 0x7fbd80073550 2024-09-22T03:46:54.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.287+0000 7fbd7cff9640 1 -- 172.21.15.3:0/3259428704 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v12) ==== 109+0+0 (crc 0 0 0) 0x7fbd700109c0 con 0x7fbd80073550 2024-09-22T03:46:54.289 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'rbd' created 2024-09-22T03:46:54.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.290+0000 7fbd864d1640 1 -- 172.21.15.3:0/3259428704 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbd6c03ea70 msgr2=0x7fbd6c040f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:54.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.290+0000 7fbd864d1640 1 --2- 172.21.15.3:0/3259428704 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbd6c03ea70 0x7fbd6c040f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:54.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.290+0000 7fbd864d1640 1 -- 172.21.15.3:0/3259428704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd80073550 msgr2=0x7fbd80123ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:54.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.290+0000 7fbd864d1640 1 --2- 172.21.15.3:0/3259428704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd80073550 0x7fbd80123ef0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:54.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.290+0000 7fbd864d1640 1 -- 172.21.15.3:0/3259428704 shutdown_connections 2024-09-22T03:46:54.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.290+0000 7fbd864d1640 1 --2- 172.21.15.3:0/3259428704 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbd6c03ea70 0x7fbd6c040f80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:54.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.290+0000 7fbd864d1640 1 --2- 172.21.15.3:0/3259428704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd80073550 0x7fbd80123ef0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:54.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.290+0000 7fbd864d1640 1 -- 172.21.15.3:0/3259428704 >> 172.21.15.3:0/3259428704 conn(0x7fbd8006f0a0 msgr2=0x7fbd80071450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:54.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.290+0000 7fbd864d1640 1 -- 172.21.15.3:0/3259428704 shutdown_connections 2024-09-22T03:46:54.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.290+0000 7fbd864d1640 1 -- 172.21.15.3:0/3259428704 wait complete. 2024-09-22T03:46:54.304 DEBUG:teuthology.orchestra.run.smithi003:> rbd --cluster ceph pool init rbd 2024-09-22T03:46:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.409+0000 7f2589e0eac0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.410+0000 7f2589e0eac0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.412+0000 7f2589e0eac0 1 Processor -- start 2024-09-22T03:46:54.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.412+0000 7f2589e0eac0 1 -- start start 2024-09-22T03:46:54.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.414+0000 7f2589e0eac0 1 --2- >> v2:172.21.15.3:3300/0 conn(0x158cb70 0x15918b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:54.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.414+0000 7f2589e0eac0 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x1357180 con 0x14f63a0 2024-09-22T03:46:54.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.414+0000 7f2589e0eac0 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x1396b90 con 0x158cb70 2024-09-22T03:46:54.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.414+0000 7f25835ab640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x158cb70 0x15918b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:54.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.414+0000 7f25835ab640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x158cb70 0x15918b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32928/0 (socket says 172.21.15.3:32928) 2024-09-22T03:46:54.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.414+0000 7f25835ab640 1 -- 172.21.15.3:0/1999829115 learned_addr learned my addr 172.21.15.3:0/1999829115 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:54.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.414+0000 7f2583dac640 1 --1- 172.21.15.3:0/1999829115 >> v1:172.21.15.3:6789/0 conn(0x14f63a0 0x158c460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49680/0 (socket says 172.21.15.3:49680) 2024-09-22T03:46:54.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.414+0000 7f25835ab640 1 -- 172.21.15.3:0/1999829115 >> v1:172.21.15.3:6789/0 conn(0x14f63a0 legacy=0x158c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:54.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.414+0000 7f25835ab640 1 -- 172.21.15.3:0/1999829115 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x132cf80 con 0x158cb70 2024-09-22T03:46:54.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.416+0000 7f25835ab640 1 --2- 172.21.15.3:0/1999829115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x158cb70 0x15918b0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994a4bc6d443254e server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:54.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.416+0000 7f2582daa640 1 -- 172.21.15.3:0/1999829115 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f256c02d840 con 0x158cb70 2024-09-22T03:46:54.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.416+0000 7f2582daa640 1 -- 172.21.15.3:0/1999829115 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f256c02d9a0 con 0x158cb70 2024-09-22T03:46:54.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.416+0000 7f2582daa640 1 -- 172.21.15.3:0/1999829115 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f256c02dc40 con 0x158cb70 2024-09-22T03:46:54.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.417+0000 7f2589e0eac0 1 -- 172.21.15.3:0/1999829115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x158cb70 msgr2=0x15918b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:54.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.417+0000 7f2589e0eac0 1 --2- 172.21.15.3:0/1999829115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x158cb70 0x15918b0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:54.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.417+0000 7f2589e0eac0 1 -- 172.21.15.3:0/1999829115 shutdown_connections 2024-09-22T03:46:54.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.417+0000 7f2589e0eac0 1 --2- 172.21.15.3:0/1999829115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x158cb70 0x15918b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:54.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.417+0000 7f2589e0eac0 1 -- 172.21.15.3:0/1999829115 >> 172.21.15.3:0/1999829115 conn(0x12fff70 msgr2=0x14f0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:54.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.417+0000 7f2589e0eac0 1 -- 172.21.15.3:0/1999829115 shutdown_connections 2024-09-22T03:46:54.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.417+0000 7f2589e0eac0 1 -- 172.21.15.3:0/1999829115 wait complete. 2024-09-22T03:46:54.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.417+0000 7f2589e0eac0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:54.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.420+0000 7f2589e0eac0 1 Processor -- start 2024-09-22T03:46:54.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.420+0000 7f2589e0eac0 1 -- start start 2024-09-22T03:46:54.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.421+0000 7f2589e0eac0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x14f63a0 0x157a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:54.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.421+0000 7f2589e0eac0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x13a18f0 con 0x14f63a0 2024-09-22T03:46:54.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.421+0000 7f2583dac640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x14f63a0 0x157a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:54.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.421+0000 7f2583dac640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x14f63a0 0x157a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32934/0 (socket says 172.21.15.3:32934) 2024-09-22T03:46:54.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.421+0000 7f2583dac640 1 -- 172.21.15.3:0/3928689887 learned_addr learned my addr 172.21.15.3:0/3928689887 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:54.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.422+0000 7f2583dac640 1 -- 172.21.15.3:0/3928689887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x133b800 con 0x14f63a0 2024-09-22T03:46:54.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.422+0000 7f2583dac640 1 --2- 172.21.15.3:0/3928689887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x14f63a0 0x157a060 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:54.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.422+0000 7f256bfff640 1 -- 172.21.15.3:0/3928689887 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f257800be50 con 0x14f63a0 2024-09-22T03:46:54.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.422+0000 7f256bfff640 1 -- 172.21.15.3:0/3928689887 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f257800c4b0 con 0x14f63a0 2024-09-22T03:46:54.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.422+0000 7f2589e0eac0 1 -- 172.21.15.3:0/3928689887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x132d310 con 0x14f63a0 2024-09-22T03:46:54.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.422+0000 7f256bfff640 1 -- 172.21.15.3:0/3928689887 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f257800c750 con 0x14f63a0 2024-09-22T03:46:54.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.423+0000 7f2589e0eac0 1 -- 172.21.15.3:0/3928689887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x132cd70 con 0x14f63a0 2024-09-22T03:46:54.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.424+0000 7f256bfff640 1 -- 172.21.15.3:0/3928689887 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 50975+0+0 (crc 0 0 0) 0x7f257800c8b0 con 0x14f63a0 2024-09-22T03:46:54.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.424+0000 7f256bfff640 1 --2- 172.21.15.3:0/3928689887 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f256403d320 0x7f256403f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:54.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.424+0000 7f256bfff640 1 -- 172.21.15.3:0/3928689887 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2661+0+0 (crc 0 0 0) 0x7f2578008040 con 0x14f63a0 2024-09-22T03:46:54.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.424+0000 7f2589e0eac0 1 -- 172.21.15.3:0/3928689887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) -- 0x157c900 con 0x14f63a0 2024-09-22T03:46:54.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.424+0000 7f25835ab640 1 --2- 172.21.15.3:0/3928689887 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f256403d320 0x7f256403f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:54.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:54.425+0000 7f25835ab640 1 --2- 172.21.15.3:0/3928689887 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f256403d320 0x7f256403f830 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:55.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.288+0000 7f256bfff640 1 -- 172.21.15.3:0/3928689887 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v13) ==== 141+0+0 (crc 0 0 0) 0x7f257800ec30 con 0x14f63a0 2024-09-22T03:46:55.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.289+0000 7f2589e0eac0 1 -- 172.21.15.3:0/3928689887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x1354520 con 0x14f63a0 2024-09-22T03:46:55.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.293+0000 7f256bfff640 1 -- 172.21.15.3:0/3928689887 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_get_version_reply(handle=1 version=13) ==== 24+0+0 (crc 0 0 0) 0x7f257800ec30 con 0x14f63a0 2024-09-22T03:46:55.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.293+0000 7f25825a9640 1 -- 172.21.15.3:0/3928689887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=13}) -- 0x7f255c002ae0 con 0x14f63a0 2024-09-22T03:46:55.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.294+0000 7f256bfff640 1 -- 172.21.15.3:0/3928689887 <== mon.0 v2:172.21.15.3:3300/0 8 ==== osd_map(13..13 src has 1..13) ==== 658+0+0 (crc 0 0 0) 0x7f2578006c70 con 0x14f63a0 2024-09-22T03:46:55.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.294+0000 7f2589e0eac0 1 --2- 172.21.15.3:0/3928689887 >> v2:172.21.15.3:6802/2784566484 conn(0x158cb70 0x157aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:55.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.294+0000 7f2589e0eac0 1 -- 172.21.15.3:0/3928689887 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13) -- 0x159f780 con 0x158cb70 2024-09-22T03:46:55.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.299+0000 7f258500e640 1 --2- 172.21.15.3:0/3928689887 >> v2:172.21.15.3:6802/2784566484 conn(0x158cb70 0x157aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T03:46:55.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.368+0000 7f258500e640 1 --2- 172.21.15.3:0/3928689887 >> v2:172.21.15.3:6802/2784566484 conn(0x158cb70 0x157aab0 crc :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:55.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.438+0000 7f258500e640 1 -- 172.21.15.3:0/3928689887 <== osd.1 v2:172.21.15.3:6802/2784566484 1 ==== osd_op_reply(1 rbd_trash [create] v13'1 uv1 ondisk = 0) ==== 153+0+0 (crc 0 0 0) 0x7f257400a4b0 con 0x158cb70 2024-09-22T03:46:55.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.443+0000 7f2589e0eac0 1 --2- 172.21.15.3:0/3928689887 >> v2:172.21.15.3:6801/2889521405 conn(0x15f2420 0x15f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:55.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.443+0000 7f2589e0eac0 1 -- 172.21.15.3:0/3928689887 --> v2:172.21.15.3:6801/2889521405 -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e13) -- 0x15f4ed0 con 0x15f2420 2024-09-22T03:46:55.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.444+0000 7f2583dac640 1 --2- 172.21.15.3:0/3928689887 >> v2:172.21.15.3:6801/2889521405 conn(0x15f2420 0x15f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T03:46:55.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.447+0000 7f2583dac640 1 --2- 172.21.15.3:0/3928689887 >> v2:172.21.15.3:6801/2889521405 conn(0x15f2420 0x15f4900 crc :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:55.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.452+0000 7f2583dac640 1 -- 172.21.15.3:0/3928689887 <== osd.0 v2:172.21.15.3:6801/2889521405 1 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f257800cc00 con 0x15f2420 2024-09-22T03:46:55.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.453+0000 7f2589e0eac0 1 -- 172.21.15.3:0/3928689887 --> v2:172.21.15.3:6801/2889521405 -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e13) -- 0x15f18e0 con 0x15f2420 2024-09-22T03:46:55.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.456+0000 7f2583dac640 1 -- 172.21.15.3:0/3928689887 <== osd.0 v2:172.21.15.3:6801/2889521405 2 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f257800cc00 con 0x15f2420 2024-09-22T03:46:55.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:55.456+0000 7f2581da8640 1 -- 172.21.15.3:0/3928689887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) -- 0x7f2560001f60 con 0x14f63a0 2024-09-22T03:46:56.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:56.301+0000 7f256bfff640 1 -- 172.21.15.3:0/3928689887 <== mon.0 v2:172.21.15.3:3300/0 9 ==== pool_op_reply(tid 4 (0) Success v14) ==== 55+0+0 (crc 0 0 0) 0x7f25780068d0 con 0x14f63a0 2024-09-22T03:46:56.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:56.301+0000 7f256bfff640 1 -- 172.21.15.3:0/3928689887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=14}) -- 0x7f2564042770 con 0x14f63a0 2024-09-22T03:46:56.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:56.302+0000 7f256bfff640 1 -- 172.21.15.3:0/3928689887 <== mon.0 v2:172.21.15.3:3300/0 10 ==== osd_map(14..14 src has 1..14) ==== 626+0+0 (crc 0 0 0) 0x7f25780066a0 con 0x14f63a0 2024-09-22T03:46:56.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:56.302+0000 7f25825a9640 1 -- 172.21.15.3:0/3928689887 --> v2:172.21.15.3:6801/2889521405 -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14) -- 0x7f255c005660 con 0x15f2420 2024-09-22T03:46:56.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:56.350+0000 7f2583dac640 1 -- 172.21.15.3:0/3928689887 <== osd.0 v2:172.21.15.3:6801/2889521405 3 ==== osd_op_reply(5 rbd_info [create,write 0~8] v14'1 uv1 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f257800cc00 con 0x15f2420 2024-09-22T03:46:56.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:56.350+0000 7f2581da8640 1 -- 172.21.15.3:0/3928689887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v14) -- 0x7f2560003370 con 0x14f63a0 2024-09-22T03:46:56.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:56.369+0000 7f256bfff640 1 -- 172.21.15.3:0/3928689887 <== mon.0 v2:172.21.15.3:3300/0 11 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f257802d5c0 con 0x14f63a0 2024-09-22T03:46:57.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.303+0000 7f256bfff640 1 -- 172.21.15.3:0/3928689887 <== mon.0 v2:172.21.15.3:3300/0 12 ==== pool_op_reply(tid 6 (0) Success v15) ==== 43+0+0 (crc 0 0 0) 0x7f2578031a50 con 0x14f63a0 2024-09-22T03:46:57.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.303+0000 7f256bfff640 1 -- 172.21.15.3:0/3928689887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=15}) -- 0x7f2564042bf0 con 0x14f63a0 2024-09-22T03:46:57.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.304+0000 7f256bfff640 1 -- 172.21.15.3:0/3928689887 <== mon.0 v2:172.21.15.3:3300/0 13 ==== osd_map(15..15 src has 1..15) ==== 654+0+0 (crc 0 0 0) 0x7f257800e380 con 0x14f63a0 2024-09-22T03:46:57.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.304+0000 7f25825a9640 1 -- 172.21.15.3:0/3928689887 --> v2:172.21.15.3:6801/2889521405 -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15) -- 0x7f255c0059c0 con 0x15f2420 2024-09-22T03:46:57.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.355+0000 7f2583dac640 1 -- 172.21.15.3:0/3928689887 <== osd.0 v2:172.21.15.3:6801/2889521405 4 ==== osd_op_reply(7 rbd_info [write 0~19] v15'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f257800cc00 con 0x15f2420 2024-09-22T03:46:57.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.355+0000 7f2589e0eac0 1 -- 172.21.15.3:0/3928689887 >> v2:172.21.15.3:6801/2889521405 conn(0x15f2420 msgr2=0x15f4900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:57.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.355+0000 7f2589e0eac0 1 --2- 172.21.15.3:0/3928689887 >> v2:172.21.15.3:6801/2889521405 conn(0x15f2420 0x15f4900 crc :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:57.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.355+0000 7f2589e0eac0 1 -- 172.21.15.3:0/3928689887 >> v2:172.21.15.3:6802/2784566484 conn(0x158cb70 msgr2=0x157aab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:57.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.355+0000 7f2589e0eac0 1 --2- 172.21.15.3:0/3928689887 >> v2:172.21.15.3:6802/2784566484 conn(0x158cb70 0x157aab0 crc :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:57.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.355+0000 7f2589e0eac0 1 -- 172.21.15.3:0/3928689887 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f256403d320 msgr2=0x7f256403f830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:57.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.355+0000 7f2589e0eac0 1 --2- 172.21.15.3:0/3928689887 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f256403d320 0x7f256403f830 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:57.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.355+0000 7f2589e0eac0 1 -- 172.21.15.3:0/3928689887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x14f63a0 msgr2=0x157a060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:57.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.355+0000 7f2589e0eac0 1 --2- 172.21.15.3:0/3928689887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x14f63a0 0x157a060 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:57.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.355+0000 7f2589e0eac0 1 -- 172.21.15.3:0/3928689887 shutdown_connections 2024-09-22T03:46:57.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.355+0000 7f2589e0eac0 1 --2- 172.21.15.3:0/3928689887 >> v2:172.21.15.3:6801/2889521405 conn(0x15f2420 0x15f4900 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:57.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.355+0000 7f2589e0eac0 1 --2- 172.21.15.3:0/3928689887 >> v2:172.21.15.3:6802/2784566484 conn(0x158cb70 0x157aab0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:57.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.355+0000 7f2589e0eac0 1 --2- 172.21.15.3:0/3928689887 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f256403d320 0x7f256403f830 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:57.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.355+0000 7f2589e0eac0 1 --2- 172.21.15.3:0/3928689887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x14f63a0 0x157a060 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:57.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.355+0000 7f2589e0eac0 1 -- 172.21.15.3:0/3928689887 >> 172.21.15.3:0/3928689887 conn(0x12fff70 msgr2=0x14fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:57.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.355+0000 7f2589e0eac0 1 -- 172.21.15.3:0/3928689887 shutdown_connections 2024-09-22T03:46:57.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.355+0000 7f2589e0eac0 1 -- 172.21.15.3:0/3928689887 wait complete. 2024-09-22T03:46:57.361 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-22T03:46:57.362 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-22T03:46:57.362 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-22T03:46:57.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.473+0000 7efc9f8d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:57.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.479+0000 7efc9f8d1640 1 Processor -- start 2024-09-22T03:46:57.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.480+0000 7efc9f8d1640 1 -- start start 2024-09-22T03:46:57.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.481+0000 7efc9f8d1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efc98133b30 0x7efc98133f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:57.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.481+0000 7efc9f8d1640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc980745c0 con 0x7efc98134480 2024-09-22T03:46:57.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.481+0000 7efc9f8d1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7efc9809e280 con 0x7efc98133b30 2024-09-22T03:46:57.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.481+0000 7efc9d66d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efc98133b30 0x7efc98133f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:57.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.481+0000 7efc9ce6c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7efc98134480 0x7efc9809c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33104/0 (socket says 172.21.15.3:33104) 2024-09-22T03:46:57.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.481+0000 7efc9ce6c640 1 -- 172.21.15.3:0/847969650 learned_addr learned my addr 172.21.15.3:0/847969650 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T03:46:57.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.481+0000 7efc9d66d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efc98133b30 0x7efc98133f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58764/0 (socket says 172.21.15.3:58764) 2024-09-22T03:46:57.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.482+0000 7efc9d66d640 1 -- 172.21.15.3:0/847969650 >> v1:172.21.15.3:6789/0 conn(0x7efc98134480 legacy=0x7efc9809c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:57.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.482+0000 7efc9d66d640 1 -- 172.21.15.3:0/847969650 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9809df70 con 0x7efc98133b30 2024-09-22T03:46:57.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.482+0000 7efc97fff640 1 -- 172.21.15.3:0/847969650 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2850810613 0 0) 0x7efc980745c0 con 0x7efc98134480 2024-09-22T03:46:57.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.482+0000 7efc9d66d640 1 --2- 172.21.15.3:0/847969650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc98133b30 0x7efc98133f40 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca84cae47e5901e server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:57.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.482+0000 7efc97fff640 1 -- 172.21.15.3:0/847969650 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efc8800e040 con 0x7efc98133b30 2024-09-22T03:46:57.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.482+0000 7efc97fff640 1 -- 172.21.15.3:0/847969650 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7efc88007670 con 0x7efc98133b30 2024-09-22T03:46:57.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.482+0000 7efc97fff640 1 -- 172.21.15.3:0/847969650 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efc88007910 con 0x7efc98133b30 2024-09-22T03:46:57.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.483+0000 7efc9f8d1640 1 -- 172.21.15.3:0/847969650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc98133b30 msgr2=0x7efc98133f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:57.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.483+0000 7efc9f8d1640 1 --2- 172.21.15.3:0/847969650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc98133b30 0x7efc98133f40 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:57.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.483+0000 7efc9f8d1640 1 -- 172.21.15.3:0/847969650 shutdown_connections 2024-09-22T03:46:57.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.483+0000 7efc9f8d1640 1 --2- 172.21.15.3:0/847969650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc98133b30 0x7efc98133f40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:57.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.483+0000 7efc9f8d1640 1 -- 172.21.15.3:0/847969650 >> 172.21.15.3:0/847969650 conn(0x7efc980a42e0 msgr2=0x7efc98073ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:57.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.483+0000 7efc9f8d1640 1 -- 172.21.15.3:0/847969650 shutdown_connections 2024-09-22T03:46:57.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.483+0000 7efc9f8d1640 1 -- 172.21.15.3:0/847969650 wait complete. 2024-09-22T03:46:57.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.483+0000 7efc9f8d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:57.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.485+0000 7efc9f8d1640 1 Processor -- start 2024-09-22T03:46:57.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.485+0000 7efc9f8d1640 1 -- start start 2024-09-22T03:46:57.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.486+0000 7efc9f8d1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc98133b30 0x7efc981334f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.486+0000 7efc9f8d1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efc980712f0 con 0x7efc98133b30 2024-09-22T03:46:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.486+0000 7efc9d66d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc98133b30 0x7efc981334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.487+0000 7efc9d66d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc98133b30 0x7efc981334f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58776/0 (socket says 172.21.15.3:58776) 2024-09-22T03:46:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.487+0000 7efc9d66d640 1 -- 172.21.15.3:0/667056323 learned_addr learned my addr 172.21.15.3:0/667056323 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.487+0000 7efc9d66d640 1 -- 172.21.15.3:0/667056323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc98125560 con 0x7efc98133b30 2024-09-22T03:46:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.487+0000 7efc9d66d640 1 --2- 172.21.15.3:0/667056323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc98133b30 0x7efc981334f0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.487+0000 7efc95ffb640 1 -- 172.21.15.3:0/667056323 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efc8800e040 con 0x7efc98133b30 2024-09-22T03:46:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.487+0000 7efc95ffb640 1 -- 172.21.15.3:0/667056323 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7efc88012b00 con 0x7efc98133b30 2024-09-22T03:46:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.487+0000 7efc9f8d1640 1 -- 172.21.15.3:0/667056323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc98070bd0 con 0x7efc98133b30 2024-09-22T03:46:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.487+0000 7efc95ffb640 1 -- 172.21.15.3:0/667056323 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efc88012da0 con 0x7efc98133b30 2024-09-22T03:46:57.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.487+0000 7efc9f8d1640 1 -- 172.21.15.3:0/667056323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc981bbc90 con 0x7efc98133b30 2024-09-22T03:46:57.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.489+0000 7efc9f8d1640 1 -- 172.21.15.3:0/667056323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc98123960 con 0x7efc98133b30 2024-09-22T03:46:57.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.489+0000 7efc95ffb640 1 -- 172.21.15.3:0/667056323 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7efc880153f0 con 0x7efc98133b30 2024-09-22T03:46:57.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.489+0000 7efc95ffb640 1 --2- 172.21.15.3:0/667056323 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7efc8403ead0 0x7efc84040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:57.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.489+0000 7efc95ffb640 1 -- 172.21.15.3:0/667056323 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7efc88051210 con 0x7efc98133b30 2024-09-22T03:46:57.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.489+0000 7efc9ce6c640 1 --2- 172.21.15.3:0/667056323 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7efc8403ead0 0x7efc84040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:57.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.490+0000 7efc9ce6c640 1 --2- 172.21.15.3:0/667056323 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7efc8403ead0 0x7efc84040fe0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:57.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.494+0000 7efc95ffb640 1 -- 172.21.15.3:0/667056323 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7efc8800e5c0 con 0x7efc98133b30 2024-09-22T03:46:57.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.682+0000 7efc9f8d1640 1 -- 172.21.15.3:0/667056323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7efc98123f80 con 0x7efc98133b30 2024-09-22T03:46:57.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.682+0000 7efc95ffb640 1 -- 172.21.15.3:0/667056323 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v1) ==== 86+0+61 (crc 0 0 0) 0x7efc880039f0 con 0x7efc98133b30 2024-09-22T03:46:57.684 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:57.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.685+0000 7efc9f8d1640 1 -- 172.21.15.3:0/667056323 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7efc8403ead0 msgr2=0x7efc84040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:57.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.685+0000 7efc9f8d1640 1 --2- 172.21.15.3:0/667056323 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7efc8403ead0 0x7efc84040fe0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:57.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.685+0000 7efc9f8d1640 1 -- 172.21.15.3:0/667056323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc98133b30 msgr2=0x7efc981334f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:57.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.685+0000 7efc9f8d1640 1 --2- 172.21.15.3:0/667056323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc98133b30 0x7efc981334f0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:57.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.685+0000 7efc9f8d1640 1 -- 172.21.15.3:0/667056323 shutdown_connections 2024-09-22T03:46:57.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.685+0000 7efc9f8d1640 1 --2- 172.21.15.3:0/667056323 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7efc8403ead0 0x7efc84040fe0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:57.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.685+0000 7efc9f8d1640 1 --2- 172.21.15.3:0/667056323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc98133b30 0x7efc981334f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:57.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.685+0000 7efc9f8d1640 1 -- 172.21.15.3:0/667056323 >> 172.21.15.3:0/667056323 conn(0x7efc980a42e0 msgr2=0x7efc981bbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:57.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.685+0000 7efc9f8d1640 1 -- 172.21.15.3:0/667056323 shutdown_connections 2024-09-22T03:46:57.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.685+0000 7efc9f8d1640 1 -- 172.21.15.3:0/667056323 wait complete. 2024-09-22T03:46:57.697 INFO:teuthology.orchestra.run.smithi003.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2024-09-22T03:46:57.698 INFO:tasks.ceph_manager:config epoch is 1 2024-09-22T03:46:57.698 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-22T03:46:57.698 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-22T03:46:57.698 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-22T03:46:57.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.806+0000 7f3106991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:57.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.811+0000 7f3106991640 1 Processor -- start 2024-09-22T03:46:57.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.811+0000 7f3106991640 1 -- start start 2024-09-22T03:46:57.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.812+0000 7f3106991640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f310009e6c0 0x7f310009c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.812+0000 7f3106991640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3100074a70 con 0x7f310009cf20 2024-09-22T03:46:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.812+0000 7f3106991640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f31000a61f0 con 0x7f310009e6c0 2024-09-22T03:46:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.813+0000 7f30fffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f310009e6c0 0x7f310009c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.813+0000 7f30fffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f310009e6c0 0x7f310009c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58788/0 (socket says 172.21.15.3:58788) 2024-09-22T03:46:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.813+0000 7f30ff7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f310009cf20 0x7f3100133ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33114/0 (socket says 172.21.15.3:33114) 2024-09-22T03:46:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.813+0000 7f30fffff640 1 -- 172.21.15.3:0/2536653519 learned_addr learned my addr 172.21.15.3:0/2536653519 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.813+0000 7f30fffff640 1 -- 172.21.15.3:0/2536653519 >> v1:172.21.15.3:6789/0 conn(0x7f310009cf20 legacy=0x7f3100133ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.813+0000 7f30feffd640 1 -- 172.21.15.3:0/2536653519 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2956119128 0 0) 0x7f3100074a70 con 0x7f310009cf20 2024-09-22T03:46:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.813+0000 7f30fffff640 1 -- 172.21.15.3:0/2536653519 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31000a5ee0 con 0x7f310009e6c0 2024-09-22T03:46:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.813+0000 7f30fffff640 1 --2- 172.21.15.3:0/2536653519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f310009e6c0 0x7f310009c9e0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe37b95cfffa02c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.813+0000 7f30feffd640 1 -- 172.21.15.3:0/2536653519 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f30f0009e40 con 0x7f310009e6c0 2024-09-22T03:46:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.813+0000 7f30feffd640 1 -- 172.21.15.3:0/2536653519 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f30f000e230 con 0x7f310009e6c0 2024-09-22T03:46:57.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.813+0000 7f30feffd640 1 -- 172.21.15.3:0/2536653519 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f30f000e4d0 con 0x7f310009e6c0 2024-09-22T03:46:57.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.814+0000 7f3106991640 1 -- 172.21.15.3:0/2536653519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f310009e6c0 msgr2=0x7f310009c9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:57.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.814+0000 7f3106991640 1 --2- 172.21.15.3:0/2536653519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f310009e6c0 0x7f310009c9e0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:57.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.814+0000 7f3106991640 1 -- 172.21.15.3:0/2536653519 shutdown_connections 2024-09-22T03:46:57.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.814+0000 7f3106991640 1 --2- 172.21.15.3:0/2536653519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f310009e6c0 0x7f310009c9e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:57.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.814+0000 7f3106991640 1 -- 172.21.15.3:0/2536653519 >> 172.21.15.3:0/2536653519 conn(0x7f31000a43e0 msgr2=0x7f3100073ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:57.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.814+0000 7f3106991640 1 -- 172.21.15.3:0/2536653519 shutdown_connections 2024-09-22T03:46:57.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.814+0000 7f3106991640 1 -- 172.21.15.3:0/2536653519 wait complete. 2024-09-22T03:46:57.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.814+0000 7f3106991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:57.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.816+0000 7f3106991640 1 Processor -- start 2024-09-22T03:46:57.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.816+0000 7f3106991640 1 -- start start 2024-09-22T03:46:57.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.818+0000 7f3106991640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f310009cf20 0x7f3100121a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:57.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.818+0000 7f3106991640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f31001234e0 con 0x7f310009cf20 2024-09-22T03:46:57.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.818+0000 7f30fffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f310009cf20 0x7f3100121a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:57.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.818+0000 7f30fffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f310009cf20 0x7f3100121a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58792/0 (socket says 172.21.15.3:58792) 2024-09-22T03:46:57.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.818+0000 7f30fffff640 1 -- 172.21.15.3:0/2120700034 learned_addr learned my addr 172.21.15.3:0/2120700034 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:57.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.818+0000 7f30fffff640 1 -- 172.21.15.3:0/2120700034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3100122bf0 con 0x7f310009cf20 2024-09-22T03:46:57.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.819+0000 7f30fffff640 1 --2- 172.21.15.3:0/2120700034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f310009cf20 0x7f3100121a60 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:57.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.819+0000 7f30fcff9640 1 -- 172.21.15.3:0/2120700034 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f30f000ee10 con 0x7f310009cf20 2024-09-22T03:46:57.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.819+0000 7f30fcff9640 1 -- 172.21.15.3:0/2120700034 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f30f0015420 con 0x7f310009cf20 2024-09-22T03:46:57.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.819+0000 7f30fcff9640 1 -- 172.21.15.3:0/2120700034 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f30f00156c0 con 0x7f310009cf20 2024-09-22T03:46:57.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.819+0000 7f3106991640 1 -- 172.21.15.3:0/2120700034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3100123d30 con 0x7f310009cf20 2024-09-22T03:46:57.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.819+0000 7f3106991640 1 -- 172.21.15.3:0/2120700034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31001244a0 con 0x7f310009cf20 2024-09-22T03:46:57.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.820+0000 7f30fcff9640 1 -- 172.21.15.3:0/2120700034 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f30f0015870 con 0x7f310009cf20 2024-09-22T03:46:57.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.820+0000 7f3106991640 1 -- 172.21.15.3:0/2120700034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31000a29d0 con 0x7f310009cf20 2024-09-22T03:46:57.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.821+0000 7f30fcff9640 1 --2- 172.21.15.3:0/2120700034 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f30ec03ead0 0x7f30ec040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:57.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.821+0000 7f30fcff9640 1 -- 172.21.15.3:0/2120700034 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7f30f0051010 con 0x7f310009cf20 2024-09-22T03:46:57.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.821+0000 7f30ff7fe640 1 --2- 172.21.15.3:0/2120700034 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f30ec03ead0 0x7f30ec040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:57.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.821+0000 7f30ff7fe640 1 --2- 172.21.15.3:0/2120700034 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f30ec03ead0 0x7f30ec040fe0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:57.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:57.826+0000 7f30fcff9640 1 -- 172.21.15.3:0/2120700034 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f31000a29d0 con 0x7f310009cf20 2024-09-22T03:46:58.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.060+0000 7f3106991640 1 -- 172.21.15.3:0/2120700034 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f31000a29d0 con 0x7f310009cf20 2024-09-22T03:46:58.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.062+0000 7f30fcff9640 1 -- 172.21.15.3:0/2120700034 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+98092 (crc 0 0 0) 0x7f31000a29d0 con 0x7f310009cf20 2024-09-22T03:46:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:58.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.065+0000 7f3106991640 1 -- 172.21.15.3:0/2120700034 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f30ec03ead0 msgr2=0x7f30ec040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.065+0000 7f3106991640 1 --2- 172.21.15.3:0/2120700034 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f30ec03ead0 0x7f30ec040fe0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.065+0000 7f3106991640 1 -- 172.21.15.3:0/2120700034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f310009cf20 msgr2=0x7f3100121a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.065+0000 7f3106991640 1 --2- 172.21.15.3:0/2120700034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f310009cf20 0x7f3100121a60 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.065+0000 7f3106991640 1 -- 172.21.15.3:0/2120700034 shutdown_connections 2024-09-22T03:46:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.065+0000 7f3106991640 1 --2- 172.21.15.3:0/2120700034 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f30ec03ead0 0x7f30ec040fe0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.065+0000 7f3106991640 1 --2- 172.21.15.3:0/2120700034 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f310009cf20 0x7f3100121a60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.065+0000 7f3106991640 1 -- 172.21.15.3:0/2120700034 >> 172.21.15.3:0/2120700034 conn(0x7f31000a43e0 msgr2=0x7f31000a0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.065+0000 7f3106991640 1 -- 172.21.15.3:0/2120700034 shutdown_connections 2024-09-22T03:46:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.065+0000 7f3106991640 1 -- 172.21.15.3:0/2120700034 wait complete. 2024-09-22T03:46:58.077 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":551784226},{"type":"v1","addr":"172.21.15.123:6801","nonce":551784226}]},"active_addr":"172.21.15.123:6801/551784226","active_change":"2024-09-22T03:46:47.547677+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":2413570255}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":3951112210}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":2235850675}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":4063541622}]}]} 2024-09-22T03:46:58.080 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-22T03:46:58.080 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-22T03:46:58.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T03:46:58.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.187+0000 7f5d6debd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:58.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.192+0000 7f5d6debd640 1 Processor -- start 2024-09-22T03:46:58.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.192+0000 7f5d6debd640 1 -- start start 2024-09-22T03:46:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.194+0000 7f5d6debd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d68070d30 0x7f5d680731f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.194+0000 7f5d6debd640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d6806fa20 con 0x7f5d68073730 2024-09-22T03:46:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.194+0000 7f5d6debd640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f5d68071710 con 0x7f5d68070d30 2024-09-22T03:46:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.194+0000 7f5d677fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5d68070d30 0x7f5d680731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.194+0000 7f5d5effd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5d68073730 0x7f5d68135da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33128/0 (socket says 172.21.15.3:33128) 2024-09-22T03:46:58.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.194+0000 7f5d5effd640 1 -- 172.21.15.3:0/913384935 learned_addr learned my addr 172.21.15.3:0/913384935 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T03:46:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.195+0000 7f5d677fe640 1 -- 172.21.15.3:0/913384935 >> v1:172.21.15.3:6789/0 conn(0x7f5d68073730 legacy=0x7f5d68135da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.195+0000 7f5d677fe640 1 -- 172.21.15.3:0/913384935 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6806ff50 con 0x7f5d68070d30 2024-09-22T03:46:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.195+0000 7f5d66ffd640 1 -- 172.21.15.3:0/913384935 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 397748341 0 0) 0x7f5d6806fa20 con 0x7f5d68073730 2024-09-22T03:46:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.195+0000 7f5d677fe640 1 --2- 172.21.15.3:0/913384935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d68070d30 0x7f5d680731f0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463bf5347bd00d51 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.195+0000 7f5d66ffd640 1 -- 172.21.15.3:0/913384935 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5d58009d80 con 0x7f5d68070d30 2024-09-22T03:46:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.195+0000 7f5d66ffd640 1 -- 172.21.15.3:0/913384935 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5d5800e230 con 0x7f5d68070d30 2024-09-22T03:46:58.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.195+0000 7f5d66ffd640 1 -- 172.21.15.3:0/913384935 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5d5800e4d0 con 0x7f5d68070d30 2024-09-22T03:46:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.196+0000 7f5d6debd640 1 -- 172.21.15.3:0/913384935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d68070d30 msgr2=0x7f5d680731f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.196+0000 7f5d6debd640 1 --2- 172.21.15.3:0/913384935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d68070d30 0x7f5d680731f0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.196+0000 7f5d6debd640 1 -- 172.21.15.3:0/913384935 shutdown_connections 2024-09-22T03:46:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.196+0000 7f5d6debd640 1 --2- 172.21.15.3:0/913384935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d68070d30 0x7f5d680731f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.196+0000 7f5d6debd640 1 -- 172.21.15.3:0/913384935 >> 172.21.15.3:0/913384935 conn(0x7f5d680a6d10 msgr2=0x7f5d68071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.196+0000 7f5d6debd640 1 -- 172.21.15.3:0/913384935 shutdown_connections 2024-09-22T03:46:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.196+0000 7f5d6debd640 1 -- 172.21.15.3:0/913384935 wait complete. 2024-09-22T03:46:58.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.196+0000 7f5d6debd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:58.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.198+0000 7f5d6debd640 1 Processor -- start 2024-09-22T03:46:58.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.198+0000 7f5d6debd640 1 -- start start 2024-09-22T03:46:58.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.199+0000 7f5d6debd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d68070d30 0x7f5d681240f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.199+0000 7f5d6debd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5d68123ca0 con 0x7f5d68070d30 2024-09-22T03:46:58.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.199+0000 7f5d677fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d68070d30 0x7f5d681240f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.199+0000 7f5d677fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d68070d30 0x7f5d681240f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58818/0 (socket says 172.21.15.3:58818) 2024-09-22T03:46:58.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.200+0000 7f5d677fe640 1 -- 172.21.15.3:0/1966111561 learned_addr learned my addr 172.21.15.3:0/1966111561 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:58.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.200+0000 7f5d677fe640 1 -- 172.21.15.3:0/1966111561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d68125120 con 0x7f5d68070d30 2024-09-22T03:46:58.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.200+0000 7f5d677fe640 1 --2- 172.21.15.3:0/1966111561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d68070d30 0x7f5d681240f0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.200+0000 7f5d64ff9640 1 -- 172.21.15.3:0/1966111561 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5d58013610 con 0x7f5d68070d30 2024-09-22T03:46:58.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.200+0000 7f5d64ff9640 1 -- 172.21.15.3:0/1966111561 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5d58013770 con 0x7f5d68070d30 2024-09-22T03:46:58.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.200+0000 7f5d6debd640 1 -- 172.21.15.3:0/1966111561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d681236a0 con 0x7f5d68070d30 2024-09-22T03:46:58.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.200+0000 7f5d64ff9640 1 -- 172.21.15.3:0/1966111561 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5d58013a10 con 0x7f5d68070d30 2024-09-22T03:46:58.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.200+0000 7f5d6debd640 1 -- 172.21.15.3:0/1966111561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d68125ff0 con 0x7f5d68070d30 2024-09-22T03:46:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.201+0000 7f5d64ff9640 1 -- 172.21.15.3:0/1966111561 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f5d58013bc0 con 0x7f5d68070d30 2024-09-22T03:46:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.201+0000 7f5d6debd640 1 -- 172.21.15.3:0/1966111561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d680731f0 con 0x7f5d68070d30 2024-09-22T03:46:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.202+0000 7f5d64ff9640 1 --2- 172.21.15.3:0/1966111561 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5d4803ead0 0x7f5d48040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.202+0000 7f5d64ff9640 1 -- 172.21.15.3:0/1966111561 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7f5d58018040 con 0x7f5d68070d30 2024-09-22T03:46:58.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.202+0000 7f5d5effd640 1 --2- 172.21.15.3:0/1966111561 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5d4803ead0 0x7f5d48040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.203+0000 7f5d5effd640 1 --2- 172.21.15.3:0/1966111561 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5d4803ead0 0x7f5d48040fe0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.207+0000 7f5d64ff9640 1 -- 172.21.15.3:0/1966111561 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f5d58011380 con 0x7f5d68070d30 2024-09-22T03:46:58.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.397+0000 7f5d6debd640 1 -- 172.21.15.3:0/1966111561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d68077360 con 0x7f5d68070d30 2024-09-22T03:46:58.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.398+0000 7f5d64ff9640 1 -- 172.21.15.3:0/1966111561 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+8859 (crc 0 0 0) 0x7f5d58018a50 con 0x7f5d68070d30 2024-09-22T03:46:58.399 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:58.399 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":15,"fsid":"012c124d-1263-4b17-8b29-e14858734658","created":"2024-09-22T03:46:43.165321+0000","modified":"2024-09-22T03:46:57.300232+0000","last_up_change":"2024-09-22T03:46:50.268789+0000","last_in_change":"2024-09-22T03:46:45.294435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T03:46:51.582570+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T03:46:53.510167+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae27f46d-f8f7-4336-ad5e-ef956da82898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6801","nonce":2889521405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2889521405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6805","nonce":2889521405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2889521405}]},"public_addr":"172.21.15.3:6801/2889521405","cluster_addr":"172.21.15.3:6800/2889521405","heartbeat_back_addr":"172.21.15.3:6805/2889521405","heartbeat_front_addr":"172.21.15.3:6804/2889521405","state":["exists","up"]},{"osd":1,"uuid":"d04614be-cd17-47ac-a9ec-7708527db672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2784566484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":2784566484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":2784566484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2784566484}]},"public_addr":"172.21.15.3:6802/2784566484","cluster_addr":"172.21.15.3:6803/2784566484","heartbeat_back_addr":"172.21.15.3:6807/2784566484","heartbeat_front_addr":"172.21.15.3:6806/2784566484","state":["exists","up"]},{"osd":2,"uuid":"8cf57c0d-5e18-448c-a419-e2d417a2d59b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":891966684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":891966684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6811","nonce":891966684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":891966684}]},"public_addr":"172.21.15.123:6813/891966684","cluster_addr":"172.21.15.123:6812/891966684","heartbeat_back_addr":"172.21.15.123:6811/891966684","heartbeat_front_addr":"172.21.15.123:6810/891966684","state":["exists","up"]},{"osd":3,"uuid":"15079b2b-ea92-49f2-8ab7-eb6edd581ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4206675901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6805","nonce":4206675901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":4206675901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4206675901}]},"public_addr":"172.21.15.123:6806/4206675901","cluster_addr":"172.21.15.123:6805/4206675901","heartbeat_back_addr":"172.21.15.123:6807/4206675901","heartbeat_front_addr":"172.21.15.123:6804/4206675901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:46:58.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.400+0000 7f5d6debd640 1 -- 172.21.15.3:0/1966111561 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5d4803ead0 msgr2=0x7f5d48040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.400+0000 7f5d6debd640 1 --2- 172.21.15.3:0/1966111561 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5d4803ead0 0x7f5d48040fe0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.400+0000 7f5d6debd640 1 -- 172.21.15.3:0/1966111561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d68070d30 msgr2=0x7f5d681240f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.400+0000 7f5d6debd640 1 --2- 172.21.15.3:0/1966111561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d68070d30 0x7f5d681240f0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.401+0000 7f5d6debd640 1 -- 172.21.15.3:0/1966111561 shutdown_connections 2024-09-22T03:46:58.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.401+0000 7f5d6debd640 1 --2- 172.21.15.3:0/1966111561 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5d4803ead0 0x7f5d48040fe0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.401+0000 7f5d6debd640 1 --2- 172.21.15.3:0/1966111561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5d68070d30 0x7f5d681240f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.401+0000 7f5d6debd640 1 -- 172.21.15.3:0/1966111561 >> 172.21.15.3:0/1966111561 conn(0x7f5d680a6d10 msgr2=0x7f5d6811cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:58.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.401+0000 7f5d6debd640 1 -- 172.21.15.3:0/1966111561 shutdown_connections 2024-09-22T03:46:58.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.401+0000 7f5d6debd640 1 -- 172.21.15.3:0/1966111561 wait complete. 2024-09-22T03:46:58.413 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-22T03:46:58.413 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T03:46:58.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.520+0000 7f34a5191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:58.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.525+0000 7f34a5191640 1 Processor -- start 2024-09-22T03:46:58.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.525+0000 7f34a5191640 1 -- start start 2024-09-22T03:46:58.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.526+0000 7f34a5191640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f34a00a4930 0x7f34a013b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.527+0000 7f34a5191640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a006ec50 con 0x7f34a00a3e10 2024-09-22T03:46:58.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.527+0000 7f34a5191640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f34a00a6130 con 0x7f34a00a4930 2024-09-22T03:46:58.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.527+0000 7f3497fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f34a00a4930 0x7f34a013b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.527+0000 7f3497fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f34a00a4930 0x7f34a013b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58824/0 (socket says 172.21.15.3:58824) 2024-09-22T03:46:58.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.527+0000 7f349ed9d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f34a00a3e10 0x7f34a00a4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33140/0 (socket says 172.21.15.3:33140) 2024-09-22T03:46:58.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.527+0000 7f3497fff640 1 -- 172.21.15.3:0/2414101141 learned_addr learned my addr 172.21.15.3:0/2414101141 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:58.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.527+0000 7f3497fff640 1 -- 172.21.15.3:0/2414101141 >> v1:172.21.15.3:6789/0 conn(0x7f34a00a3e10 legacy=0x7f34a00a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.527+0000 7f3497fff640 1 -- 172.21.15.3:0/2414101141 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a00a5110 con 0x7f34a00a4930 2024-09-22T03:46:58.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.527+0000 7f3497fff640 1 --2- 172.21.15.3:0/2414101141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a00a4930 0x7f34a013b2d0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f583d6336fe81d server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.527+0000 7f349e59c640 1 -- 172.21.15.3:0/2414101141 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3488009e50 con 0x7f34a00a4930 2024-09-22T03:46:58.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.527+0000 7f349e59c640 1 -- 172.21.15.3:0/2414101141 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f348800e230 con 0x7f34a00a4930 2024-09-22T03:46:58.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.527+0000 7f349e59c640 1 -- 172.21.15.3:0/2414101141 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f348800e4d0 con 0x7f34a00a4930 2024-09-22T03:46:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.528+0000 7f34a5191640 1 -- 172.21.15.3:0/2414101141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a00a4930 msgr2=0x7f34a013b2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.528+0000 7f34a5191640 1 --2- 172.21.15.3:0/2414101141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a00a4930 0x7f34a013b2d0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.528+0000 7f34a5191640 1 -- 172.21.15.3:0/2414101141 shutdown_connections 2024-09-22T03:46:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.528+0000 7f34a5191640 1 --2- 172.21.15.3:0/2414101141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a00a4930 0x7f34a013b2d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.528+0000 7f34a5191640 1 -- 172.21.15.3:0/2414101141 >> 172.21.15.3:0/2414101141 conn(0x7f34a00a74f0 msgr2=0x7f34a00a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.528+0000 7f34a5191640 1 -- 172.21.15.3:0/2414101141 shutdown_connections 2024-09-22T03:46:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.528+0000 7f34a5191640 1 -- 172.21.15.3:0/2414101141 wait complete. 2024-09-22T03:46:58.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.528+0000 7f34a5191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.530+0000 7f34a5191640 1 Processor -- start 2024-09-22T03:46:58.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.530+0000 7f34a5191640 1 -- start start 2024-09-22T03:46:58.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.532+0000 7f34a5191640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a00a3e10 0x7f34a0123920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.532+0000 7f34a5191640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01253a0 con 0x7f34a00a3e10 2024-09-22T03:46:58.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.532+0000 7f349ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a00a3e10 0x7f34a0123920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.532+0000 7f349ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a00a3e10 0x7f34a0123920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58838/0 (socket says 172.21.15.3:58838) 2024-09-22T03:46:58.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.532+0000 7f349ed9d640 1 -- 172.21.15.3:0/1982278614 learned_addr learned my addr 172.21.15.3:0/1982278614 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:58.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.532+0000 7f349ed9d640 1 -- 172.21.15.3:0/1982278614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a0124b60 con 0x7f34a00a3e10 2024-09-22T03:46:58.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.533+0000 7f349ed9d640 1 --2- 172.21.15.3:0/1982278614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a00a3e10 0x7f34a0123920 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.533+0000 7f34977fe640 1 -- 172.21.15.3:0/1982278614 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f34900059a0 con 0x7f34a00a3e10 2024-09-22T03:46:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.533+0000 7f34a5191640 1 -- 172.21.15.3:0/1982278614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a0125c50 con 0x7f34a00a3e10 2024-09-22T03:46:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.533+0000 7f34977fe640 1 -- 172.21.15.3:0/1982278614 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3490005b00 con 0x7f34a00a3e10 2024-09-22T03:46:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.533+0000 7f34977fe640 1 -- 172.21.15.3:0/1982278614 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3490005da0 con 0x7f34a00a3e10 2024-09-22T03:46:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.533+0000 7f34a5191640 1 -- 172.21.15.3:0/1982278614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a01263d0 con 0x7f34a00a3e10 2024-09-22T03:46:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.534+0000 7f34a5191640 1 -- 172.21.15.3:0/1982278614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a01277f0 con 0x7f34a00a3e10 2024-09-22T03:46:58.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.534+0000 7f34977fe640 1 -- 172.21.15.3:0/1982278614 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f34900070b0 con 0x7f34a00a3e10 2024-09-22T03:46:58.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.534+0000 7f34977fe640 1 --2- 172.21.15.3:0/1982278614 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f348003ea80 0x7f3480040f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.534+0000 7f34977fe640 1 -- 172.21.15.3:0/1982278614 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7f349004f8e0 con 0x7f34a00a3e10 2024-09-22T03:46:58.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.534+0000 7f3497fff640 1 --2- 172.21.15.3:0/1982278614 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f348003ea80 0x7f3480040f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.535+0000 7f3497fff640 1 --2- 172.21.15.3:0/1982278614 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f348003ea80 0x7f3480040f90 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.539+0000 7f34977fe640 1 -- 172.21.15.3:0/1982278614 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f349000dc60 con 0x7f34a00a3e10 2024-09-22T03:46:58.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.727+0000 7f34a5191640 1 -- 172.21.15.3:0/1982278614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34a006e050 con 0x7f34a00a3e10 2024-09-22T03:46:58.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.728+0000 7f34977fe640 1 -- 172.21.15.3:0/1982278614 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+8859 (crc 0 0 0) 0x7f349000d660 con 0x7f34a00a3e10 2024-09-22T03:46:58.729 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:46:58.729 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":15,"fsid":"012c124d-1263-4b17-8b29-e14858734658","created":"2024-09-22T03:46:43.165321+0000","modified":"2024-09-22T03:46:57.300232+0000","last_up_change":"2024-09-22T03:46:50.268789+0000","last_in_change":"2024-09-22T03:46:45.294435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T03:46:51.582570+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T03:46:53.510167+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae27f46d-f8f7-4336-ad5e-ef956da82898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6801","nonce":2889521405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2889521405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6805","nonce":2889521405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2889521405}]},"public_addr":"172.21.15.3:6801/2889521405","cluster_addr":"172.21.15.3:6800/2889521405","heartbeat_back_addr":"172.21.15.3:6805/2889521405","heartbeat_front_addr":"172.21.15.3:6804/2889521405","state":["exists","up"]},{"osd":1,"uuid":"d04614be-cd17-47ac-a9ec-7708527db672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2784566484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":2784566484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":2784566484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2784566484}]},"public_addr":"172.21.15.3:6802/2784566484","cluster_addr":"172.21.15.3:6803/2784566484","heartbeat_back_addr":"172.21.15.3:6807/2784566484","heartbeat_front_addr":"172.21.15.3:6806/2784566484","state":["exists","up"]},{"osd":2,"uuid":"8cf57c0d-5e18-448c-a419-e2d417a2d59b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":891966684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":891966684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6811","nonce":891966684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":891966684}]},"public_addr":"172.21.15.123:6813/891966684","cluster_addr":"172.21.15.123:6812/891966684","heartbeat_back_addr":"172.21.15.123:6811/891966684","heartbeat_front_addr":"172.21.15.123:6810/891966684","state":["exists","up"]},{"osd":3,"uuid":"15079b2b-ea92-49f2-8ab7-eb6edd581ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4206675901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6805","nonce":4206675901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":4206675901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4206675901}]},"public_addr":"172.21.15.123:6806/4206675901","cluster_addr":"172.21.15.123:6805/4206675901","heartbeat_back_addr":"172.21.15.123:6807/4206675901","heartbeat_front_addr":"172.21.15.123:6804/4206675901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T03:46:58.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.730+0000 7f34a5191640 1 -- 172.21.15.3:0/1982278614 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f348003ea80 msgr2=0x7f3480040f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.730+0000 7f34a5191640 1 --2- 172.21.15.3:0/1982278614 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f348003ea80 0x7f3480040f90 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.730+0000 7f34a5191640 1 -- 172.21.15.3:0/1982278614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a00a3e10 msgr2=0x7f34a0123920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.730+0000 7f34a5191640 1 --2- 172.21.15.3:0/1982278614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a00a3e10 0x7f34a0123920 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.730+0000 7f34a5191640 1 -- 172.21.15.3:0/1982278614 shutdown_connections 2024-09-22T03:46:58.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.730+0000 7f34a5191640 1 --2- 172.21.15.3:0/1982278614 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f348003ea80 0x7f3480040f90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.730+0000 7f34a5191640 1 --2- 172.21.15.3:0/1982278614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a00a3e10 0x7f34a0123920 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.730+0000 7f34a5191640 1 -- 172.21.15.3:0/1982278614 >> 172.21.15.3:0/1982278614 conn(0x7f34a00a74f0 msgr2=0x7f34a0071470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:58.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.730+0000 7f34a5191640 1 -- 172.21.15.3:0/1982278614 shutdown_connections 2024-09-22T03:46:58.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.730+0000 7f34a5191640 1 -- 172.21.15.3:0/1982278614 wait complete. 2024-09-22T03:46:58.743 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-22T03:46:58.743 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-22T03:46:58.744 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-22T03:46:58.744 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-22T03:46:58.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.862+0000 7f75b78bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:58.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.862+0000 7f424bed1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:58.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.862+0000 7f92884d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:58.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.862+0000 7fa3f966c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:58.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.868+0000 7f75b78bd640 1 Processor -- start 2024-09-22T03:46:58.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.868+0000 7f92884d1640 1 Processor -- start 2024-09-22T03:46:58.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.869+0000 7f92884d1640 1 -- start start 2024-09-22T03:46:58.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.869+0000 7f75b78bd640 1 -- start start 2024-09-22T03:46:58.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.869+0000 7f424bed1640 1 Processor -- start 2024-09-22T03:46:58.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.869+0000 7f424bed1640 1 -- start start 2024-09-22T03:46:58.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.869+0000 7fa3f966c640 1 Processor -- start 2024-09-22T03:46:58.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.869+0000 7fa3f966c640 1 -- start start 2024-09-22T03:46:58.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.870+0000 7f75b78bd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f75b00a4930 0x7f75b013b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.870+0000 7f75b78bd640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75b006ec50 con 0x7f75b00a3e10 2024-09-22T03:46:58.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.870+0000 7f92884d1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f92800a3e10 0x7f92800a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.870+0000 7f75b78bd640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f75b00a6130 con 0x7f75b00a4930 2024-09-22T03:46:58.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f92884d1640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f928006ec50 con 0x7f92800a4760 2024-09-22T03:46:58.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f75b5659640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f75b00a3e10 0x7f75b00a4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33148/0 (socket says 172.21.15.3:33148) 2024-09-22T03:46:58.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f75b5659640 1 -- 172.21.15.3:0/3576758215 learned_addr learned my addr 172.21.15.3:0/3576758215 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T03:46:58.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f92884d1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f92800a6130 con 0x7f92800a3e10 2024-09-22T03:46:58.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f424bed1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f42440a4930 0x7f424413b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f928626d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f92800a3e10 0x7f92800a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f75b4e58640 1 --2- 172.21.15.3:0/3576758215 >> v2:172.21.15.3:3300/0 conn(0x7f75b00a4930 0x7f75b013b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f9285a6c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f92800a4760 0x7f9280135e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33156/0 (socket says 172.21.15.3:33156) 2024-09-22T03:46:58.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f424bed1640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f424406ec50 con 0x7f42440a3e10 2024-09-22T03:46:58.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7fa3f966c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3f40a3e10 0x7fa3f40a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f424bed1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f42440a6130 con 0x7f42440a4930 2024-09-22T03:46:58.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f9285a6c640 1 -- 172.21.15.3:0/3435069219 learned_addr learned my addr 172.21.15.3:0/3435069219 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T03:46:58.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f424946c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f42440a4930 0x7f424413b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f4249c6d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f42440a3e10 0x7f42440a4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33160/0 (socket says 172.21.15.3:33160) 2024-09-22T03:46:58.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f424946c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f42440a4930 0x7f424413b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58874/0 (socket says 172.21.15.3:58874) 2024-09-22T03:46:58.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f4249c6d640 1 -- 172.21.15.3:0/3614315296 learned_addr learned my addr 172.21.15.3:0/3614315296 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T03:46:58.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7fa3f2ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3f40a3e10 0x7fa3f40a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f75a7fff640 1 -- 172.21.15.3:0/3576758215 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4071771527 0 0) 0x7f75b006ec50 con 0x7f75b00a3e10 2024-09-22T03:46:58.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f75a7fff640 1 -- 172.21.15.3:0/3576758215 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f759c003580 con 0x7f75b00a3e10 2024-09-22T03:46:58.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7fa3f2ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa3f40a3e10 0x7fa3f40a4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58888/0 (socket says 172.21.15.3:58888) 2024-09-22T03:46:58.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f75b4e58640 1 -- 172.21.15.3:0/3576758215 >> v1:172.21.15.3:6789/0 conn(0x7f75b00a3e10 legacy=0x7f75b00a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f75b4e58640 1 -- 172.21.15.3:0/3576758215 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b00a5110 con 0x7f75b00a4930 2024-09-22T03:46:58.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7fa3f2ffd640 1 -- 172.21.15.3:0/1822022480 learned_addr learned my addr 172.21.15.3:0/1822022480 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:58.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7fa3f27fc640 1 --1- 172.21.15.3:0/1822022480 >> v1:172.21.15.3:6789/0 conn(0x7fa3f40a4760 0x7fa3f4135e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33162/0 (socket says 172.21.15.3:33162) 2024-09-22T03:46:58.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7fa3f966c640 1 -- 172.21.15.3:0/1822022480 --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3f406ec50 con 0x7fa3f40a4760 2024-09-22T03:46:58.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7fa3f966c640 1 -- 172.21.15.3:0/1822022480 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa3f40a6130 con 0x7fa3f40a3e10 2024-09-22T03:46:58.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.871+0000 7f928526b640 1 -- 172.21.15.3:0/3435069219 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 325369116 0 0) 0x7f928006ec50 con 0x7f92800a4760 2024-09-22T03:46:58.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7f928526b640 1 -- 172.21.15.3:0/3435069219 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f926c003580 con 0x7f92800a4760 2024-09-22T03:46:58.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7f424946c640 1 -- 172.21.15.3:0/3614315296 >> v1:172.21.15.3:6789/0 conn(0x7f42440a3e10 legacy=0x7f42440a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7f424946c640 1 -- 172.21.15.3:0/3614315296 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42440a5110 con 0x7f42440a4930 2024-09-22T03:46:58.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7f928626d640 1 -- 172.21.15.3:0/3435069219 >> v1:172.21.15.3:6789/0 conn(0x7f92800a4760 legacy=0x7f9280135e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7f928626d640 1 -- 172.21.15.3:0/3435069219 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92800a5110 con 0x7f92800a3e10 2024-09-22T03:46:58.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7f75b4e58640 1 --2- 172.21.15.3:0/3576758215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b00a4930 0x7f75b013b2d0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502b9bd8fd5c708f server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7fa3f1ffb640 1 -- 172.21.15.3:0/1822022480 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1351326662 0 0) 0x7fa3f406ec50 con 0x7fa3f40a4760 2024-09-22T03:46:58.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7fa3f1ffb640 1 -- 172.21.15.3:0/1822022480 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3dc003580 con 0x7fa3f40a4760 2024-09-22T03:46:58.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7f928626d640 1 --2- 172.21.15.3:0/3435069219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92800a3e10 0x7f92800a4220 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93c022ea07927b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7fa3f2ffd640 1 -- 172.21.15.3:0/1822022480 >> v1:172.21.15.3:6789/0 conn(0x7fa3f40a4760 legacy=0x7fa3f4135e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7fa3f2ffd640 1 -- 172.21.15.3:0/1822022480 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f40a5110 con 0x7fa3f40a3e10 2024-09-22T03:46:58.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7f75a7fff640 1 -- 172.21.15.3:0/3576758215 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f75a0009d90 con 0x7f75b00a4930 2024-09-22T03:46:58.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7f928526b640 1 -- 172.21.15.3:0/3435069219 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f927c009d90 con 0x7f92800a3e10 2024-09-22T03:46:58.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7f928526b640 1 -- 172.21.15.3:0/3435069219 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f927c00e230 con 0x7f92800a3e10 2024-09-22T03:46:58.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7f928526b640 1 -- 172.21.15.3:0/3435069219 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f927c00e4d0 con 0x7f92800a3e10 2024-09-22T03:46:58.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7f424946c640 1 --2- 172.21.15.3:0/3614315296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42440a4930 0x7f424413b2d0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2deb48daaec8d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7f75a7fff640 1 -- 172.21.15.3:0/3576758215 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f75a000e230 con 0x7f75b00a4930 2024-09-22T03:46:58.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7f75a7fff640 1 -- 172.21.15.3:0/3576758215 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f75a000e4d0 con 0x7f75b00a4930 2024-09-22T03:46:58.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7fa3f2ffd640 1 --2- 172.21.15.3:0/1822022480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f40a3e10 0x7fa3f40a4220 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e32965dfd9deee server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7f4248c6b640 1 -- 172.21.15.3:0/3614315296 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4234009d90 con 0x7f42440a4930 2024-09-22T03:46:58.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7f4248c6b640 1 -- 172.21.15.3:0/3614315296 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f423400e230 con 0x7f42440a4930 2024-09-22T03:46:58.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.872+0000 7f4248c6b640 1 -- 172.21.15.3:0/3614315296 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f423400e4d0 con 0x7f42440a4930 2024-09-22T03:46:58.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.873+0000 7fa3f1ffb640 1 -- 172.21.15.3:0/1822022480 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa3e4009e50 con 0x7fa3f40a3e10 2024-09-22T03:46:58.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.873+0000 7fa3f1ffb640 1 -- 172.21.15.3:0/1822022480 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa3e400e230 con 0x7fa3f40a3e10 2024-09-22T03:46:58.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.873+0000 7fa3f1ffb640 1 -- 172.21.15.3:0/1822022480 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa3e400e4d0 con 0x7fa3f40a3e10 2024-09-22T03:46:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.873+0000 7f92884d1640 1 -- 172.21.15.3:0/3435069219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92800a3e10 msgr2=0x7f92800a4220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.873+0000 7f92884d1640 1 --2- 172.21.15.3:0/3435069219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92800a3e10 0x7f92800a4220 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.873+0000 7f92884d1640 1 -- 172.21.15.3:0/3435069219 shutdown_connections 2024-09-22T03:46:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.873+0000 7f92884d1640 1 --2- 172.21.15.3:0/3435069219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92800a3e10 0x7f92800a4220 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.873+0000 7f92884d1640 1 -- 172.21.15.3:0/3435069219 >> 172.21.15.3:0/3435069219 conn(0x7f92800a74f0 msgr2=0x7f92800a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:58.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.873+0000 7f75b78bd640 1 -- 172.21.15.3:0/3576758215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b00a4930 msgr2=0x7f75b013b2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.873+0000 7f75b78bd640 1 --2- 172.21.15.3:0/3576758215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b00a4930 0x7f75b013b2d0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.873+0000 7f92884d1640 1 -- 172.21.15.3:0/3435069219 shutdown_connections 2024-09-22T03:46:58.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.873+0000 7f92884d1640 1 -- 172.21.15.3:0/3435069219 wait complete. 2024-09-22T03:46:58.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.873+0000 7f424bed1640 1 -- 172.21.15.3:0/3614315296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42440a4930 msgr2=0x7f424413b2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.873+0000 7f424bed1640 1 --2- 172.21.15.3:0/3614315296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42440a4930 0x7f424413b2d0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.873+0000 7f75b78bd640 1 -- 172.21.15.3:0/3576758215 shutdown_connections 2024-09-22T03:46:58.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.873+0000 7f75b78bd640 1 --2- 172.21.15.3:0/3576758215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b00a4930 0x7f75b013b2d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.873+0000 7f75b78bd640 1 -- 172.21.15.3:0/3576758215 >> 172.21.15.3:0/3576758215 conn(0x7f75b00a74f0 msgr2=0x7f75b00a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:58.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7fa3f966c640 1 -- 172.21.15.3:0/1822022480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f40a3e10 msgr2=0x7fa3f40a4220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7fa3f966c640 1 --2- 172.21.15.3:0/1822022480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f40a3e10 0x7fa3f40a4220 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7f75b78bd640 1 -- 172.21.15.3:0/3576758215 shutdown_connections 2024-09-22T03:46:58.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7f424bed1640 1 -- 172.21.15.3:0/3614315296 shutdown_connections 2024-09-22T03:46:58.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7f424bed1640 1 --2- 172.21.15.3:0/3614315296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42440a4930 0x7f424413b2d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7f424bed1640 1 -- 172.21.15.3:0/3614315296 >> 172.21.15.3:0/3614315296 conn(0x7f42440a74f0 msgr2=0x7f42440a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:58.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7f92884d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:58.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7f75b78bd640 1 -- 172.21.15.3:0/3576758215 wait complete. 2024-09-22T03:46:58.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7fa3f966c640 1 -- 172.21.15.3:0/1822022480 shutdown_connections 2024-09-22T03:46:58.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7fa3f966c640 1 --2- 172.21.15.3:0/1822022480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f40a3e10 0x7fa3f40a4220 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7fa3f966c640 1 -- 172.21.15.3:0/1822022480 >> 172.21.15.3:0/1822022480 conn(0x7fa3f40a74f0 msgr2=0x7fa3f40a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:58.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7f424bed1640 1 -- 172.21.15.3:0/3614315296 shutdown_connections 2024-09-22T03:46:58.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7f424bed1640 1 -- 172.21.15.3:0/3614315296 wait complete. 2024-09-22T03:46:58.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7fa3f966c640 1 -- 172.21.15.3:0/1822022480 shutdown_connections 2024-09-22T03:46:58.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7fa3f966c640 1 -- 172.21.15.3:0/1822022480 wait complete. 2024-09-22T03:46:58.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7f75b78bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:58.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7f424bed1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:58.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.874+0000 7fa3f966c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:58.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.876+0000 7f75b78bd640 1 Processor -- start 2024-09-22T03:46:58.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.876+0000 7f75b78bd640 1 -- start start 2024-09-22T03:46:58.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.876+0000 7fa3f966c640 1 Processor -- start 2024-09-22T03:46:58.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.876+0000 7f92884d1640 1 Processor -- start 2024-09-22T03:46:58.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.876+0000 7fa3f966c640 1 -- start start 2024-09-22T03:46:58.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.876+0000 7f92884d1640 1 -- start start 2024-09-22T03:46:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.877+0000 7f424bed1640 1 Processor -- start 2024-09-22T03:46:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.877+0000 7f424bed1640 1 -- start start 2024-09-22T03:46:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.878+0000 7f75b78bd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b00a3e10 0x7f75b01ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.878+0000 7f75b78bd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f75b0127770 con 0x7f75b00a3e10 2024-09-22T03:46:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.878+0000 7f75b5659640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b00a3e10 0x7f75b01ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.878+0000 7f75b5659640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b00a3e10 0x7f75b01ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58896/0 (socket says 172.21.15.3:58896) 2024-09-22T03:46:58.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.878+0000 7f75b5659640 1 -- 172.21.15.3:0/3218068442 learned_addr learned my addr 172.21.15.3:0/3218068442 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:58.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.878+0000 7fa3f966c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f40a3e10 0x7fa3f4123df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.878+0000 7fa3f966c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4126070 con 0x7fa3f40a3e10 2024-09-22T03:46:58.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.878+0000 7f75b5659640 1 -- 172.21.15.3:0/3218068442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b01261c0 con 0x7f75b00a3e10 2024-09-22T03:46:58.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.878+0000 7fa3f2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f40a3e10 0x7fa3f4123df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.878+0000 7fa3f2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f40a3e10 0x7fa3f4123df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58900/0 (socket says 172.21.15.3:58900) 2024-09-22T03:46:58.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.878+0000 7fa3f2ffd640 1 -- 172.21.15.3:0/3678454057 learned_addr learned my addr 172.21.15.3:0/3678454057 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:58.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.878+0000 7f75b5659640 1 --2- 172.21.15.3:0/3218068442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b00a3e10 0x7f75b01ca630 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.879+0000 7fa3f2ffd640 1 -- 172.21.15.3:0/3678454057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f41251c0 con 0x7fa3f40a3e10 2024-09-22T03:46:58.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.879+0000 7f75a5ffb640 1 -- 172.21.15.3:0/3218068442 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f75ac004b80 con 0x7f75b00a3e10 2024-09-22T03:46:58.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.879+0000 7f75a5ffb640 1 -- 172.21.15.3:0/3218068442 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f75ac004ce0 con 0x7f75b00a3e10 2024-09-22T03:46:58.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.879+0000 7f75b78bd640 1 -- 172.21.15.3:0/3218068442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b0125f10 con 0x7f75b00a3e10 2024-09-22T03:46:58.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.879+0000 7f92884d1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92800a3e10 0x7f9280123df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.879+0000 7fa3f2ffd640 1 --2- 172.21.15.3:0/3678454057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f40a3e10 0x7fa3f4123df0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.879+0000 7f92884d1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9280126070 con 0x7f92800a3e10 2024-09-22T03:46:58.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.879+0000 7f75a5ffb640 1 -- 172.21.15.3:0/3218068442 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f75ac007120 con 0x7f75b00a3e10 2024-09-22T03:46:58.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.879+0000 7f75b78bd640 1 -- 172.21.15.3:0/3218068442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b01237d0 con 0x7f75b00a3e10 2024-09-22T03:46:58.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.879+0000 7fa3db7fe640 1 -- 172.21.15.3:0/3678454057 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa3e400ec60 con 0x7fa3f40a3e10 2024-09-22T03:46:58.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.879+0000 7fa3f966c640 1 -- 172.21.15.3:0/3678454057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f4123780 con 0x7fa3f40a3e10 2024-09-22T03:46:58.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.879+0000 7fa3f966c640 1 -- 172.21.15.3:0/3678454057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f4123a50 con 0x7fa3f40a3e10 2024-09-22T03:46:58.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.879+0000 7fa3db7fe640 1 -- 172.21.15.3:0/3678454057 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa3e400edc0 con 0x7fa3f40a3e10 2024-09-22T03:46:58.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.879+0000 7fa3db7fe640 1 -- 172.21.15.3:0/3678454057 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa3e400ec60 con 0x7fa3f40a3e10 2024-09-22T03:46:58.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.880+0000 7f928626d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92800a3e10 0x7f9280123df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.880+0000 7f928626d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92800a3e10 0x7f9280123df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58904/0 (socket says 172.21.15.3:58904) 2024-09-22T03:46:58.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.880+0000 7f928626d640 1 -- 172.21.15.3:0/2018406094 learned_addr learned my addr 172.21.15.3:0/2018406094 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:58.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.880+0000 7f75b78bd640 1 -- 172.21.15.3:0/3218068442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f75b00a5e20 con 0x7f75b00a3e10 2024-09-22T03:46:58.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.880+0000 7f424bed1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42440a3e10 0x7f4244123f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.880+0000 7f424bed1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f42441260a0 con 0x7f42440a3e10 2024-09-22T03:46:58.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.880+0000 7fa3f966c640 1 -- 172.21.15.3:0/3678454057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3d4000f80 con 0x7fa3f40a3e10 2024-09-22T03:46:58.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.880+0000 7f4249c6d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42440a3e10 0x7f4244123f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.880+0000 7f4249c6d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42440a3e10 0x7f4244123f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58908/0 (socket says 172.21.15.3:58908) 2024-09-22T03:46:58.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.880+0000 7f4249c6d640 1 -- 172.21.15.3:0/3610463373 learned_addr learned my addr 172.21.15.3:0/3610463373 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:58.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.880+0000 7f928626d640 1 -- 172.21.15.3:0/2018406094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92801251c0 con 0x7f92800a3e10 2024-09-22T03:46:58.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.881+0000 7f75a5ffb640 1 -- 172.21.15.3:0/3218068442 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f75ac007280 con 0x7f75b00a3e10 2024-09-22T03:46:58.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.881+0000 7f928626d640 1 --2- 172.21.15.3:0/2018406094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92800a3e10 0x7f9280123df0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.881+0000 7f9276ffd640 1 -- 172.21.15.3:0/2018406094 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f927c014410 con 0x7f92800a3e10 2024-09-22T03:46:58.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.881+0000 7f92884d1640 1 -- 172.21.15.3:0/2018406094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9280123780 con 0x7f92800a3e10 2024-09-22T03:46:58.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.881+0000 7f9276ffd640 1 -- 172.21.15.3:0/2018406094 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f927c014570 con 0x7f92800a3e10 2024-09-22T03:46:58.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.881+0000 7f9276ffd640 1 -- 172.21.15.3:0/2018406094 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f927c014810 con 0x7f92800a3e10 2024-09-22T03:46:58.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.881+0000 7fa3db7fe640 1 -- 172.21.15.3:0/3678454057 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fa3e4015420 con 0x7fa3f40a3e10 2024-09-22T03:46:58.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.881+0000 7f92884d1640 1 -- 172.21.15.3:0/2018406094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9280123a50 con 0x7f92800a3e10 2024-09-22T03:46:58.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.881+0000 7f75a5ffb640 1 --2- 172.21.15.3:0/3218068442 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f759403ead0 0x7f7594040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.881+0000 7f75a5ffb640 1 -- 172.21.15.3:0/3218068442 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7f75ac04fa10 con 0x7f75b00a3e10 2024-09-22T03:46:58.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.881+0000 7f4249c6d640 1 -- 172.21.15.3:0/3610463373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4244123bd0 con 0x7f42440a3e10 2024-09-22T03:46:58.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.881+0000 7f4249c6d640 1 --2- 172.21.15.3:0/3610463373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42440a3e10 0x7f4244123f70 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.881+0000 7fa3db7fe640 1 --2- 172.21.15.3:0/3678454057 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa3e003eaf0 0x7fa3e0041000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.881+0000 7f423a7fc640 1 -- 172.21.15.3:0/3610463373 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4240005a60 con 0x7f42440a3e10 2024-09-22T03:46:58.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.881+0000 7f423a7fc640 1 -- 172.21.15.3:0/3610463373 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4240005bc0 con 0x7f42440a3e10 2024-09-22T03:46:58.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.882+0000 7f75a5ffb640 1 --2- 172.21.15.3:0/3218068442 >> v2:172.21.15.123:6813/891966684 conn(0x7f7594043ad0 0x7f7594045fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.882+0000 7f424bed1640 1 -- 172.21.15.3:0/3610463373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4244125b10 con 0x7f42440a3e10 2024-09-22T03:46:58.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.882+0000 7f423a7fc640 1 -- 172.21.15.3:0/3610463373 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4240005e60 con 0x7f42440a3e10 2024-09-22T03:46:58.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.882+0000 7f75a5ffb640 1 -- 172.21.15.3:0/3218068442 --> v2:172.21.15.123:6813/891966684 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7594046630 con 0x7f7594043ad0 2024-09-22T03:46:58.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.882+0000 7f75a5ffb640 1 -- 172.21.15.3:0/3218068442 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (crc 0 0 0) 0x7f75ac04fda0 con 0x7f75b00a3e10 2024-09-22T03:46:58.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.882+0000 7fa3db7fe640 1 -- 172.21.15.3:0/3678454057 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7fa3e4010890 con 0x7fa3f40a3e10 2024-09-22T03:46:58.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.882+0000 7f424bed1640 1 -- 172.21.15.3:0/3610463373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4244126ce0 con 0x7f42440a3e10 2024-09-22T03:46:58.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.882+0000 7fa3db7fe640 1 --2- 172.21.15.3:0/3678454057 >> v2:172.21.15.3:6802/2784566484 conn(0x7fa3e0044300 0x7fa3e00467f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.882+0000 7fa3db7fe640 1 -- 172.21.15.3:0/3678454057 --> v2:172.21.15.3:6802/2784566484 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3e0046e80 con 0x7fa3e0044300 2024-09-22T03:46:58.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.882+0000 7fa3f27fc640 1 --2- 172.21.15.3:0/3678454057 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa3e003eaf0 0x7fa3e0041000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.882+0000 7f75b4e58640 1 --2- 172.21.15.3:0/3218068442 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f759403ead0 0x7f7594040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.882+0000 7f92884d1640 1 -- 172.21.15.3:0/2018406094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9280127850 con 0x7f92800a3e10 2024-09-22T03:46:58.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.882+0000 7fa3db7fe640 1 -- 172.21.15.3:0/3678454057 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (crc 0 0 0) 0x7fa3e40094f0 con 0x7fa3f40a3e10 2024-09-22T03:46:58.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.883+0000 7fa3f27fc640 1 --2- 172.21.15.3:0/3678454057 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa3e003eaf0 0x7fa3e0041000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.883+0000 7f75b4e58640 1 --2- 172.21.15.3:0/3218068442 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f759403ead0 0x7f7594040fe0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.883+0000 7fa3f37fe640 1 --2- 172.21.15.3:0/3678454057 >> v2:172.21.15.3:6802/2784566484 conn(0x7fa3e0044300 0x7fa3e00467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T03:46:58.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.883+0000 7f75b5e5a640 1 --2- 172.21.15.3:0/3218068442 >> v2:172.21.15.123:6813/891966684 conn(0x7f7594043ad0 0x7f7594045fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T03:46:58.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.883+0000 7f9276ffd640 1 -- 172.21.15.3:0/2018406094 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f927c014a10 con 0x7f92800a3e10 2024-09-22T03:46:58.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.883+0000 7f423a7fc640 1 -- 172.21.15.3:0/3610463373 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f424000c7f0 con 0x7f42440a3e10 2024-09-22T03:46:58.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.884+0000 7f423a7fc640 1 --2- 172.21.15.3:0/3610463373 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f422803eb00 0x7f4228041010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.884+0000 7f9276ffd640 1 --2- 172.21.15.3:0/2018406094 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f926403eb00 0x7f9264041010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.884+0000 7f423a7fc640 1 -- 172.21.15.3:0/3610463373 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7f424004f930 con 0x7f42440a3e10 2024-09-22T03:46:58.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.884+0000 7f9276ffd640 1 -- 172.21.15.3:0/2018406094 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7f927c019040 con 0x7f92800a3e10 2024-09-22T03:46:58.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.884+0000 7f424bed1640 1 -- 172.21.15.3:0/3610463373 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f42440a5e20 con 0x7f42440a3e10 2024-09-22T03:46:58.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.884+0000 7f9276ffd640 1 --2- 172.21.15.3:0/2018406094 >> v2:172.21.15.123:6806/4206675901 conn(0x7f9264043b00 0x7f9264045fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.884+0000 7f9276ffd640 1 -- 172.21.15.3:0/2018406094 --> v2:172.21.15.123:6806/4206675901 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9264046660 con 0x7f9264043b00 2024-09-22T03:46:58.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.884+0000 7f9285a6c640 1 --2- 172.21.15.3:0/2018406094 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f926403eb00 0x7f9264041010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.884+0000 7f9276ffd640 1 -- 172.21.15.3:0/2018406094 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (crc 0 0 0) 0x7f927c0534a0 con 0x7f92800a3e10 2024-09-22T03:46:58.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.884+0000 7f423a7fc640 1 --2- 172.21.15.3:0/3610463373 >> v2:172.21.15.3:6801/2889521405 conn(0x7f4228043b00 0x7f4228045fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:58.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.884+0000 7f423a7fc640 1 -- 172.21.15.3:0/3610463373 --> v2:172.21.15.3:6801/2889521405 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4228046660 con 0x7f4228043b00 2024-09-22T03:46:58.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.884+0000 7f423a7fc640 1 -- 172.21.15.3:0/3610463373 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (crc 0 0 0) 0x7f424000f230 con 0x7f42440a3e10 2024-09-22T03:46:58.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.884+0000 7f424946c640 1 --2- 172.21.15.3:0/3610463373 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f422803eb00 0x7f4228041010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:58.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.885+0000 7f9285a6c640 1 --2- 172.21.15.3:0/2018406094 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f926403eb00 0x7f9264041010 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.885+0000 7f424946c640 1 --2- 172.21.15.3:0/3610463373 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f422803eb00 0x7f4228041010 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.885+0000 7f9286a6e640 1 --2- 172.21.15.3:0/2018406094 >> v2:172.21.15.123:6806/4206675901 conn(0x7f9264043b00 0x7f9264045fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T03:46:58.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.885+0000 7f424a46e640 1 --2- 172.21.15.3:0/3610463373 >> v2:172.21.15.3:6801/2889521405 conn(0x7f4228043b00 0x7f4228045fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T03:46:58.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.888+0000 7f75b5e5a640 1 --2- 172.21.15.3:0/3218068442 >> v2:172.21.15.123:6813/891966684 conn(0x7f7594043ad0 0x7f7594045fa0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.888+0000 7f9286a6e640 1 --2- 172.21.15.3:0/2018406094 >> v2:172.21.15.123:6806/4206675901 conn(0x7f9264043b00 0x7f9264045fd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.888+0000 7fa3f37fe640 1 --2- 172.21.15.3:0/3678454057 >> v2:172.21.15.3:6802/2784566484 conn(0x7fa3e0044300 0x7fa3e00467f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.889+0000 7f424a46e640 1 --2- 172.21.15.3:0/3610463373 >> v2:172.21.15.3:6801/2889521405 conn(0x7f4228043b00 0x7f4228045fd0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:58.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.889+0000 7f9276ffd640 1 -- 172.21.15.3:0/2018406094 <== osd.3 v2:172.21.15.123:6806/4206675901 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5790 (crc 0 0 0) 0x7f9264046660 con 0x7f9264043b00 2024-09-22T03:46:58.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.889+0000 7f75a5ffb640 1 -- 172.21.15.3:0/3218068442 <== osd.2 v2:172.21.15.123:6813/891966684 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5790 (crc 0 0 0) 0x7f7594046630 con 0x7f7594043ad0 2024-09-22T03:46:58.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.890+0000 7fa3db7fe640 1 -- 172.21.15.3:0/3678454057 <== osd.1 v2:172.21.15.3:6802/2784566484 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5790 (crc 0 0 0) 0x7fa3e0046e80 con 0x7fa3e0044300 2024-09-22T03:46:58.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.891+0000 7f423a7fc640 1 -- 172.21.15.3:0/3610463373 <== osd.0 v2:172.21.15.3:6801/2889521405 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5790 (crc 0 0 0) 0x7f4228046660 con 0x7f4228043b00 2024-09-22T03:46:58.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.894+0000 7f75b78bd640 1 -- 172.21.15.3:0/3218068442 --> v2:172.21.15.123:6813/891966684 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f75b0125b30 con 0x7f7594043ad0 2024-09-22T03:46:58.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.894+0000 7f424bed1640 1 -- 172.21.15.3:0/3610463373 --> v2:172.21.15.3:6801/2889521405 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4244125ef0 con 0x7f4228043b00 2024-09-22T03:46:58.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.895+0000 7f75a5ffb640 1 -- 172.21.15.3:0/3218068442 <== osd.2 v2:172.21.15.123:6813/891966684 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f75b0125b30 con 0x7f7594043ad0 2024-09-22T03:46:58.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.896+0000 7f75b78bd640 1 -- 172.21.15.3:0/3218068442 >> v2:172.21.15.123:6813/891966684 conn(0x7f7594043ad0 msgr2=0x7f7594045fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.896+0000 7f75b78bd640 1 --2- 172.21.15.3:0/3218068442 >> v2:172.21.15.123:6813/891966684 conn(0x7f7594043ad0 0x7f7594045fa0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.896+0000 7f423a7fc640 1 -- 172.21.15.3:0/3610463373 <== osd.0 v2:172.21.15.3:6801/2889521405 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4244125ef0 con 0x7f4228043b00 2024-09-22T03:46:58.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.896+0000 7f424bed1640 1 -- 172.21.15.3:0/3610463373 >> v2:172.21.15.3:6801/2889521405 conn(0x7f4228043b00 msgr2=0x7f4228045fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.896+0000 7f75b78bd640 1 -- 172.21.15.3:0/3218068442 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f759403ead0 msgr2=0x7f7594040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.896+0000 7f75b78bd640 1 --2- 172.21.15.3:0/3218068442 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f759403ead0 0x7f7594040fe0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.896+0000 7f92884d1640 1 -- 172.21.15.3:0/2018406094 --> v2:172.21.15.123:6806/4206675901 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f92800a5e20 con 0x7f9264043b00 2024-09-22T03:46:58.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.896+0000 7f424bed1640 1 --2- 172.21.15.3:0/3610463373 >> v2:172.21.15.3:6801/2889521405 conn(0x7f4228043b00 0x7f4228045fd0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.896+0000 7f75b78bd640 1 -- 172.21.15.3:0/3218068442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b00a3e10 msgr2=0x7f75b01ca630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.896+0000 7f75b78bd640 1 --2- 172.21.15.3:0/3218068442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b00a3e10 0x7f75b01ca630 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f424bed1640 1 -- 172.21.15.3:0/3610463373 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f422803eb00 msgr2=0x7f4228041010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f424bed1640 1 --2- 172.21.15.3:0/3610463373 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f422803eb00 0x7f4228041010 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f75b78bd640 1 -- 172.21.15.3:0/3218068442 shutdown_connections 2024-09-22T03:46:58.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f75b78bd640 1 --2- 172.21.15.3:0/3218068442 >> v2:172.21.15.123:6813/891966684 conn(0x7f7594043ad0 0x7f7594045fa0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f424bed1640 1 -- 172.21.15.3:0/3610463373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42440a3e10 msgr2=0x7f4244123f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f75b78bd640 1 --2- 172.21.15.3:0/3218068442 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f759403ead0 0x7f7594040fe0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f424bed1640 1 --2- 172.21.15.3:0/3610463373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42440a3e10 0x7f4244123f70 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f75b78bd640 1 --2- 172.21.15.3:0/3218068442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75b00a3e10 0x7f75b01ca630 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f75b78bd640 1 -- 172.21.15.3:0/3218068442 >> 172.21.15.3:0/3218068442 conn(0x7f75b00a74f0 msgr2=0x7f75b00713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:58.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f424bed1640 1 -- 172.21.15.3:0/3610463373 shutdown_connections 2024-09-22T03:46:58.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f424bed1640 1 --2- 172.21.15.3:0/3610463373 >> v2:172.21.15.3:6801/2889521405 conn(0x7f4228043b00 0x7f4228045fd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f75b78bd640 1 -- 172.21.15.3:0/3218068442 shutdown_connections 2024-09-22T03:46:58.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f424bed1640 1 --2- 172.21.15.3:0/3610463373 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f422803eb00 0x7f4228041010 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f424bed1640 1 --2- 172.21.15.3:0/3610463373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42440a3e10 0x7f4244123f70 crc :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f75b78bd640 1 -- 172.21.15.3:0/3218068442 wait complete. 2024-09-22T03:46:58.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f424bed1640 1 -- 172.21.15.3:0/3610463373 >> 172.21.15.3:0/3610463373 conn(0x7f42440a74f0 msgr2=0x7f4244071360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:58.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f424bed1640 1 -- 172.21.15.3:0/3610463373 shutdown_connections 2024-09-22T03:46:58.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.897+0000 7f424bed1640 1 -- 172.21.15.3:0/3610463373 wait complete. 2024-09-22T03:46:58.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.898+0000 7f9276ffd640 1 -- 172.21.15.3:0/2018406094 <== osd.3 v2:172.21.15.123:6806/4206675901 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f927800a820 con 0x7f9264043b00 2024-09-22T03:46:58.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.898+0000 7fa3f966c640 1 -- 172.21.15.3:0/3678454057 --> v2:172.21.15.3:6802/2784566484 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa3d4002c30 con 0x7fa3e0044300 2024-09-22T03:46:58.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.898+0000 7f9274ff9640 1 -- 172.21.15.3:0/2018406094 >> v2:172.21.15.123:6806/4206675901 conn(0x7f9264043b00 msgr2=0x7f9264045fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.898+0000 7f9274ff9640 1 --2- 172.21.15.3:0/2018406094 >> v2:172.21.15.123:6806/4206675901 conn(0x7f9264043b00 0x7f9264045fd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.898+0000 7f9274ff9640 1 -- 172.21.15.3:0/2018406094 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f926403eb00 msgr2=0x7f9264041010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.898+0000 7f9274ff9640 1 --2- 172.21.15.3:0/2018406094 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f926403eb00 0x7f9264041010 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.899+0000 7f9274ff9640 1 -- 172.21.15.3:0/2018406094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92800a3e10 msgr2=0x7f9280123df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.899+0000 7f9274ff9640 1 --2- 172.21.15.3:0/2018406094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92800a3e10 0x7f9280123df0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.899+0000 7f9274ff9640 1 -- 172.21.15.3:0/2018406094 shutdown_connections 2024-09-22T03:46:58.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.899+0000 7f9274ff9640 1 --2- 172.21.15.3:0/2018406094 >> v2:172.21.15.123:6806/4206675901 conn(0x7f9264043b00 0x7f9264045fd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.899+0000 7f9274ff9640 1 --2- 172.21.15.3:0/2018406094 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f926403eb00 0x7f9264041010 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.899+0000 7f9274ff9640 1 --2- 172.21.15.3:0/2018406094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92800a3e10 0x7f9280123df0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.899+0000 7f9274ff9640 1 -- 172.21.15.3:0/2018406094 >> 172.21.15.3:0/2018406094 conn(0x7f92800a74f0 msgr2=0x7f92800713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:58.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.899+0000 7f9274ff9640 1 -- 172.21.15.3:0/2018406094 shutdown_connections 2024-09-22T03:46:58.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.899+0000 7f9274ff9640 1 -- 172.21.15.3:0/2018406094 wait complete. 2024-09-22T03:46:58.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.900+0000 7fa3db7fe640 1 -- 172.21.15.3:0/3678454057 <== osd.1 v2:172.21.15.3:6802/2784566484 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa3d4002c30 con 0x7fa3e0044300 2024-09-22T03:46:58.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.901+0000 7fa3f966c640 1 -- 172.21.15.3:0/3678454057 >> v2:172.21.15.3:6802/2784566484 conn(0x7fa3e0044300 msgr2=0x7fa3e00467f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.901+0000 7fa3f966c640 1 --2- 172.21.15.3:0/3678454057 >> v2:172.21.15.3:6802/2784566484 conn(0x7fa3e0044300 0x7fa3e00467f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.901+0000 7fa3f966c640 1 -- 172.21.15.3:0/3678454057 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa3e003eaf0 msgr2=0x7fa3e0041000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.901+0000 7fa3f966c640 1 --2- 172.21.15.3:0/3678454057 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa3e003eaf0 0x7fa3e0041000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.901+0000 7fa3f966c640 1 -- 172.21.15.3:0/3678454057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f40a3e10 msgr2=0x7fa3f4123df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:58.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.901+0000 7fa3f966c640 1 --2- 172.21.15.3:0/3678454057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f40a3e10 0x7fa3f4123df0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.901+0000 7fa3f966c640 1 -- 172.21.15.3:0/3678454057 shutdown_connections 2024-09-22T03:46:58.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.901+0000 7fa3f966c640 1 --2- 172.21.15.3:0/3678454057 >> v2:172.21.15.3:6802/2784566484 conn(0x7fa3e0044300 0x7fa3e00467f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.901+0000 7fa3f966c640 1 --2- 172.21.15.3:0/3678454057 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa3e003eaf0 0x7fa3e0041000 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.901+0000 7fa3f966c640 1 --2- 172.21.15.3:0/3678454057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3f40a3e10 0x7fa3f4123df0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:58.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.901+0000 7fa3f966c640 1 -- 172.21.15.3:0/3678454057 >> 172.21.15.3:0/3678454057 conn(0x7fa3f40a74f0 msgr2=0x7fa3f40713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:58.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.902+0000 7fa3f966c640 1 -- 172.21.15.3:0/3678454057 shutdown_connections 2024-09-22T03:46:58.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:58.902+0000 7fa3f966c640 1 -- 172.21.15.3:0/3678454057 wait complete. 2024-09-22T03:46:58.929 INFO:teuthology.orchestra.run.smithi003.stdout:38654705672 2024-09-22T03:46:58.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T03:46:58.930 INFO:teuthology.orchestra.run.smithi003.stdout:38654705672 2024-09-22T03:46:58.931 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T03:46:58.931 INFO:teuthology.orchestra.run.smithi003.stdout:38654705672 2024-09-22T03:46:58.931 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T03:46:58.932 INFO:teuthology.orchestra.run.smithi003.stdout:38654705672 2024-09-22T03:46:58.933 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T03:46:59.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.088+0000 7f0fab85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:59.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.088+0000 7f98fab11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:59.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.088+0000 7f3306791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:59.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.089+0000 7f5645791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:59.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.095+0000 7f0fab85a640 1 Processor -- start 2024-09-22T03:46:59.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.095+0000 7f0fab85a640 1 -- start start 2024-09-22T03:46:59.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.095+0000 7f98fab11640 1 Processor -- start 2024-09-22T03:46:59.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.095+0000 7f3306791640 1 Processor -- start 2024-09-22T03:46:59.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.095+0000 7f98fab11640 1 -- start start 2024-09-22T03:46:59.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.095+0000 7f3306791640 1 -- start start 2024-09-22T03:46:59.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.095+0000 7f5645791640 1 Processor -- start 2024-09-22T03:46:59.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.096+0000 7f5645791640 1 -- start start 2024-09-22T03:46:59.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f0fab85a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0fa40a3e10 0x7f0fa40a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:59.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f0fab85a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fa406ec50 con 0x7f0fa40a4760 2024-09-22T03:46:59.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f0fab85a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0fa40a6130 con 0x7f0fa40a3e10 2024-09-22T03:46:59.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f0fa95f6640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0fa40a3e10 0x7f0fa40a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:59.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f98fab11640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f98f4136e60 0x7f98f4139280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:59.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f0fa95f6640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0fa40a3e10 0x7f0fa40a4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58916/0 (socket says 172.21.15.3:58916) 2024-09-22T03:46:59.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f0fa8df5640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0fa40a4760 0x7f0fa4135e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33164/0 (socket says 172.21.15.3:33164) 2024-09-22T03:46:59.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f0fa8df5640 1 -- 172.21.15.3:0/1766670634 learned_addr learned my addr 172.21.15.3:0/1766670634 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T03:46:59.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f98fab11640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98f409deb0 con 0x7f98f4134010 2024-09-22T03:46:59.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f98fab11640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f98f40a5b20 con 0x7f98f4136e60 2024-09-22T03:46:59.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f98f88ad640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f98f4134010 0x7f98f4134420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33180/0 (socket says 172.21.15.3:33180) 2024-09-22T03:46:59.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f98f88ad640 1 -- 172.21.15.3:0/2064745924 learned_addr learned my addr 172.21.15.3:0/2064745924 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T03:46:59.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f98f3fff640 1 --2- 172.21.15.3:0/2064745924 >> v2:172.21.15.3:3300/0 conn(0x7f98f4136e60 0x7f98f4139280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:59.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f3306791640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3300134010 0x7f3300134420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:59.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f0fa95f6640 1 -- 172.21.15.3:0/1766670634 >> v1:172.21.15.3:6789/0 conn(0x7f0fa40a4760 legacy=0x7f0fa4135e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:59.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f0f9bfff640 1 -- 172.21.15.3:0/1766670634 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 248480844 0 0) 0x7f0fa406ec50 con 0x7f0fa40a4760 2024-09-22T03:46:59.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f0fa95f6640 1 -- 172.21.15.3:0/1766670634 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa40a5110 con 0x7f0fa40a3e10 2024-09-22T03:46:59.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f3306791640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f330009deb0 con 0x7f330009c9e0 2024-09-22T03:46:59.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f5645791640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5640135b10 0x7f5640137f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:59.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.097+0000 7f3306791640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f33000a5b20 con 0x7f3300134010 2024-09-22T03:46:59.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f5645791640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56401317c0 con 0x7f5640131c10 2024-09-22T03:46:59.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f5645791640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f564009e120 con 0x7f5640135b10 2024-09-22T03:46:59.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f563effd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5640131c10 0x7f5640132020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33202/0 (socket says 172.21.15.3:33202) 2024-09-22T03:46:59.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f563effd640 1 -- 172.21.15.3:0/2533872802 learned_addr learned my addr 172.21.15.3:0/2533872802 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T03:46:59.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f98f3fff640 1 -- 172.21.15.3:0/2064745924 >> v1:172.21.15.3:6789/0 conn(0x7f98f4134010 legacy=0x7f98f4134420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:59.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f98f3fff640 1 -- 172.21.15.3:0/2064745924 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f40a5cf0 con 0x7f98f4136e60 2024-09-22T03:46:59.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f563e7fc640 1 --2- 172.21.15.3:0/2533872802 >> v2:172.21.15.3:3300/0 conn(0x7f5640135b10 0x7f5640137f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:59.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f32ff7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f330009c9e0 0x7f330009cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33186/0 (socket says 172.21.15.3:33186) 2024-09-22T03:46:59.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f32ff7fe640 1 -- 172.21.15.3:0/3849868170 learned_addr learned my addr 172.21.15.3:0/3849868170 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T03:46:59.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f32fffff640 1 --2- 172.21.15.3:0/3849868170 >> v2:172.21.15.3:3300/0 conn(0x7f3300134010 0x7f3300134420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:59.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f0fa95f6640 1 --2- 172.21.15.3:0/1766670634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa40a3e10 0x7f0fa40a4220 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e718971fe2748cc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:59.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f98f3fff640 1 --2- 172.21.15.3:0/2064745924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f4136e60 0x7f98f4139280 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80dcb7c731da443f server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:59.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f32feffd640 1 -- 172.21.15.3:0/3849868170 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 16605826 0 0) 0x7f330009deb0 con 0x7f330009c9e0 2024-09-22T03:46:59.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f32feffd640 1 -- 172.21.15.3:0/3849868170 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32e8003580 con 0x7f330009c9e0 2024-09-22T03:46:59.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f0f9bfff640 1 -- 172.21.15.3:0/1766670634 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0fa0009d90 con 0x7f0fa40a3e10 2024-09-22T03:46:59.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f563e7fc640 1 -- 172.21.15.3:0/2533872802 >> v1:172.21.15.3:6789/0 conn(0x7f5640131c10 legacy=0x7f5640132020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:59.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f563e7fc640 1 -- 172.21.15.3:0/2533872802 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5640070220 con 0x7f5640135b10 2024-09-22T03:46:59.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f98f37fe640 1 -- 172.21.15.3:0/2064745924 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f98e0009d90 con 0x7f98f4136e60 2024-09-22T03:46:59.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f32fffff640 1 -- 172.21.15.3:0/3849868170 >> v1:172.21.15.3:6789/0 conn(0x7f330009c9e0 legacy=0x7f330009cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:59.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f32fffff640 1 -- 172.21.15.3:0/3849868170 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33000a5cf0 con 0x7f3300134010 2024-09-22T03:46:59.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.098+0000 7f563e7fc640 1 --2- 172.21.15.3:0/2533872802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5640135b10 0x7f5640137f30 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a866f3325c379c61 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:59.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.099+0000 7f0f9bfff640 1 -- 172.21.15.3:0/1766670634 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0fa000e230 con 0x7f0fa40a3e10 2024-09-22T03:46:59.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.099+0000 7f0f9bfff640 1 -- 172.21.15.3:0/1766670634 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0fa000e4d0 con 0x7f0fa40a3e10 2024-09-22T03:46:59.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.099+0000 7f98f37fe640 1 -- 172.21.15.3:0/2064745924 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f98e000e230 con 0x7f98f4136e60 2024-09-22T03:46:59.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.099+0000 7f98f37fe640 1 -- 172.21.15.3:0/2064745924 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f98e000e4d0 con 0x7f98f4136e60 2024-09-22T03:46:59.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.099+0000 7f32fffff640 1 --2- 172.21.15.3:0/3849868170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3300134010 0x7f3300134420 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d534c9fd24a24551 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:59.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.099+0000 7f563dffb640 1 -- 172.21.15.3:0/2533872802 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5634009d90 con 0x7f5640135b10 2024-09-22T03:46:59.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.099+0000 7f563dffb640 1 -- 172.21.15.3:0/2533872802 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f563400e230 con 0x7f5640135b10 2024-09-22T03:46:59.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.099+0000 7f563dffb640 1 -- 172.21.15.3:0/2533872802 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f563400e4d0 con 0x7f5640135b10 2024-09-22T03:46:59.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.099+0000 7f32feffd640 1 -- 172.21.15.3:0/3849868170 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f32f4001620 con 0x7f3300134010 2024-09-22T03:46:59.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.099+0000 7f32feffd640 1 -- 172.21.15.3:0/3849868170 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f32f0009b50 con 0x7f3300134010 2024-09-22T03:46:59.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.099+0000 7f32feffd640 1 -- 172.21.15.3:0/3849868170 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f32f0009df0 con 0x7f3300134010 2024-09-22T03:46:59.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f5645791640 1 -- 172.21.15.3:0/2533872802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5640135b10 msgr2=0x7f5640137f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:59.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f5645791640 1 --2- 172.21.15.3:0/2533872802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5640135b10 0x7f5640137f30 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f0fab85a640 1 -- 172.21.15.3:0/1766670634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa40a3e10 msgr2=0x7f0fa40a4220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:59.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f0fab85a640 1 --2- 172.21.15.3:0/1766670634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa40a3e10 0x7f0fa40a4220 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f5645791640 1 -- 172.21.15.3:0/2533872802 shutdown_connections 2024-09-22T03:46:59.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f5645791640 1 --2- 172.21.15.3:0/2533872802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5640135b10 0x7f5640137f30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f5645791640 1 -- 172.21.15.3:0/2533872802 >> 172.21.15.3:0/2533872802 conn(0x7f564006e770 msgr2=0x7f564006eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:59.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f0fab85a640 1 -- 172.21.15.3:0/1766670634 shutdown_connections 2024-09-22T03:46:59.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f3306791640 1 -- 172.21.15.3:0/3849868170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3300134010 msgr2=0x7f3300134420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:59.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f3306791640 1 --2- 172.21.15.3:0/3849868170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3300134010 0x7f3300134420 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f98fab11640 1 -- 172.21.15.3:0/2064745924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f4136e60 msgr2=0x7f98f4139280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:59.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f98fab11640 1 --2- 172.21.15.3:0/2064745924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f4136e60 0x7f98f4139280 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f0fab85a640 1 --2- 172.21.15.3:0/1766670634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa40a3e10 0x7f0fa40a4220 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f0fab85a640 1 -- 172.21.15.3:0/1766670634 >> 172.21.15.3:0/1766670634 conn(0x7f0fa40a74f0 msgr2=0x7f0fa40a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:59.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f3306791640 1 -- 172.21.15.3:0/3849868170 shutdown_connections 2024-09-22T03:46:59.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f3306791640 1 --2- 172.21.15.3:0/3849868170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3300134010 0x7f3300134420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f98fab11640 1 -- 172.21.15.3:0/2064745924 shutdown_connections 2024-09-22T03:46:59.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f3306791640 1 -- 172.21.15.3:0/3849868170 >> 172.21.15.3:0/3849868170 conn(0x7f33000a43e0 msgr2=0x7f3300073ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:59.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f5645791640 1 -- 172.21.15.3:0/2533872802 shutdown_connections 2024-09-22T03:46:59.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f5645791640 1 -- 172.21.15.3:0/2533872802 wait complete. 2024-09-22T03:46:59.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f98fab11640 1 --2- 172.21.15.3:0/2064745924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f4136e60 0x7f98f4139280 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f98fab11640 1 -- 172.21.15.3:0/2064745924 >> 172.21.15.3:0/2064745924 conn(0x7f98f40a43e0 msgr2=0x7f98f4073ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:59.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f98fab11640 1 -- 172.21.15.3:0/2064745924 shutdown_connections 2024-09-22T03:46:59.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f3306791640 1 -- 172.21.15.3:0/3849868170 shutdown_connections 2024-09-22T03:46:59.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f98fab11640 1 -- 172.21.15.3:0/2064745924 wait complete. 2024-09-22T03:46:59.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f0fab85a640 1 -- 172.21.15.3:0/1766670634 shutdown_connections 2024-09-22T03:46:59.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f3306791640 1 -- 172.21.15.3:0/3849868170 wait complete. 2024-09-22T03:46:59.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f0fab85a640 1 -- 172.21.15.3:0/1766670634 wait complete. 2024-09-22T03:46:59.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.100+0000 7f5645791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:59.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.101+0000 7f98fab11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:59.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.101+0000 7f0fab85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:59.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.101+0000 7f3306791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:46:59.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.102+0000 7f98fab11640 1 Processor -- start 2024-09-22T03:46:59.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.102+0000 7f98fab11640 1 -- start start 2024-09-22T03:46:59.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.102+0000 7f0fab85a640 1 Processor -- start 2024-09-22T03:46:59.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.102+0000 7f0fab85a640 1 -- start start 2024-09-22T03:46:59.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.103+0000 7f3306791640 1 Processor -- start 2024-09-22T03:46:59.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.103+0000 7f5645791640 1 Processor -- start 2024-09-22T03:46:59.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.103+0000 7f5645791640 1 -- start start 2024-09-22T03:46:59.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.104+0000 7f3306791640 1 -- start start 2024-09-22T03:46:59.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.104+0000 7f0fab85a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa40a3e10 0x7f0fa4123df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:59.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.104+0000 7f0fab85a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa4126070 con 0x7f0fa40a3e10 2024-09-22T03:46:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.104+0000 7f98fab11640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f4134010 0x7f98f4121080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.104+0000 7f98fab11640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f98f4120cc0 con 0x7f98f4134010 2024-09-22T03:46:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.104+0000 7f0fa95f6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa40a3e10 0x7f0fa4123df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.104+0000 7f0fa95f6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa40a3e10 0x7f0fa4123df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58946/0 (socket says 172.21.15.3:58946) 2024-09-22T03:46:59.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.104+0000 7f0fa95f6640 1 -- 172.21.15.3:0/2372675189 learned_addr learned my addr 172.21.15.3:0/2372675189 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.104+0000 7f98f88ad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f4134010 0x7f98f4121080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.104+0000 7f98f88ad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f4134010 0x7f98f4121080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58954/0 (socket says 172.21.15.3:58954) 2024-09-22T03:46:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.104+0000 7f98f88ad640 1 -- 172.21.15.3:0/1237867726 learned_addr learned my addr 172.21.15.3:0/1237867726 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.105+0000 7f0fa95f6640 1 -- 172.21.15.3:0/2372675189 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa41251c0 con 0x7f0fa40a3e10 2024-09-22T03:46:59.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.105+0000 7f98f88ad640 1 -- 172.21.15.3:0/1237867726 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f4122010 con 0x7f98f4134010 2024-09-22T03:46:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.105+0000 7f0fa95f6640 1 --2- 172.21.15.3:0/2372675189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa40a3e10 0x7f0fa4123df0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:59.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.105+0000 7f98f88ad640 1 --2- 172.21.15.3:0/1237867726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f4134010 0x7f98f4121080 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:59.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.105+0000 7f0f99ffb640 1 -- 172.21.15.3:0/2372675189 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0fa0013610 con 0x7f0fa40a3e10 2024-09-22T03:46:59.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.105+0000 7f0f99ffb640 1 -- 172.21.15.3:0/2372675189 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0fa0013770 con 0x7f0fa40a3e10 2024-09-22T03:46:59.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.105+0000 7f0fab85a640 1 -- 172.21.15.3:0/2372675189 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa4123780 con 0x7f0fa40a3e10 2024-09-22T03:46:59.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.105+0000 7f0f99ffb640 1 -- 172.21.15.3:0/2372675189 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0fa0013a10 con 0x7f0fa40a3e10 2024-09-22T03:46:59.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.105+0000 7f98f17fa640 1 -- 172.21.15.3:0/1237867726 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f98e8005a60 con 0x7f98f4134010 2024-09-22T03:46:59.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.105+0000 7f98f17fa640 1 -- 172.21.15.3:0/1237867726 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f98e8005bc0 con 0x7f98f4134010 2024-09-22T03:46:59.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.106+0000 7f0fab85a640 1 -- 172.21.15.3:0/2372675189 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa4123a50 con 0x7f0fa40a3e10 2024-09-22T03:46:59.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.105+0000 7f98fab11640 1 -- 172.21.15.3:0/1237867726 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f4122f70 con 0x7f98f4134010 2024-09-22T03:46:59.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.106+0000 7f5645791640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5640131c10 0x7f564011ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:59.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.106+0000 7f5645791640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f564011f000 con 0x7f5640131c10 2024-09-22T03:46:59.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.106+0000 7f3306791640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f330009c9e0 0x7f330011e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:59.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.106+0000 7f3306791640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3300132bb0 con 0x7f330009c9e0 2024-09-22T03:46:59.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.106+0000 7f563effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5640131c10 0x7f564011ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:59.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.106+0000 7f563effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5640131c10 0x7f564011ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58970/0 (socket says 172.21.15.3:58970) 2024-09-22T03:46:59.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.106+0000 7f563effd640 1 -- 172.21.15.3:0/4105127720 learned_addr learned my addr 172.21.15.3:0/4105127720 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:59.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.106+0000 7f32fffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f330009c9e0 0x7f330011e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:59.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.106+0000 7f32fffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f330009c9e0 0x7f330011e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58956/0 (socket says 172.21.15.3:58956) 2024-09-22T03:46:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.107+0000 7f32fffff640 1 -- 172.21.15.3:0/1480892727 learned_addr learned my addr 172.21.15.3:0/1480892727 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:46:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.107+0000 7f32fffff640 1 -- 172.21.15.3:0/1480892727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3300122980 con 0x7f330009c9e0 2024-09-22T03:46:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.107+0000 7f98f17fa640 1 -- 172.21.15.3:0/1237867726 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f98e8005e60 con 0x7f98f4134010 2024-09-22T03:46:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.107+0000 7f98fab11640 1 -- 172.21.15.3:0/1237867726 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f4120920 con 0x7f98f4134010 2024-09-22T03:46:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.107+0000 7f563effd640 1 -- 172.21.15.3:0/4105127720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5640120d20 con 0x7f5640131c10 2024-09-22T03:46:59.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.107+0000 7f0f99ffb640 1 -- 172.21.15.3:0/2372675189 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f0fa0013c10 con 0x7f0fa40a3e10 2024-09-22T03:46:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.107+0000 7f0fab85a640 1 -- 172.21.15.3:0/2372675189 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa4125ae0 con 0x7f0fa40a3e10 2024-09-22T03:46:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.107+0000 7f0f99ffb640 1 --2- 172.21.15.3:0/2372675189 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f0f8803eab0 0x7f0f88040fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.107+0000 7f0f99ffb640 1 -- 172.21.15.3:0/2372675189 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7f0fa0018040 con 0x7f0fa40a3e10 2024-09-22T03:46:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.107+0000 7f32fffff640 1 --2- 172.21.15.3:0/1480892727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f330009c9e0 0x7f330011e970 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.107+0000 7f0fa8df5640 1 --2- 172.21.15.3:0/2372675189 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f0f8803eab0 0x7f0f88040fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:59.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.107+0000 7f563effd640 1 --2- 172.21.15.3:0/4105127720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5640131c10 0x7f564011ccd0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.107+0000 7f98f17fa640 1 -- 172.21.15.3:0/1237867726 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f98e800c7f0 con 0x7f98f4134010 2024-09-22T03:46:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.108+0000 7f98f17fa640 1 --2- 172.21.15.3:0/1237867726 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f98d803ead0 0x7f98d8040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.108+0000 7f98fab11640 1 -- 172.21.15.3:0/1237867726 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98f406ab50 con 0x7f98f4134010 2024-09-22T03:46:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.108+0000 7f0fa8df5640 1 --2- 172.21.15.3:0/2372675189 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f0f8803eab0 0x7f0f88040fc0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.108+0000 7f98f3fff640 1 --2- 172.21.15.3:0/1237867726 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f98d803ead0 0x7f98d8040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.109+0000 7f98f3fff640 1 --2- 172.21.15.3:0/1237867726 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f98d803ead0 0x7f98d8040fe0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.113+0000 7f32fcff9640 1 -- 172.21.15.3:0/1480892727 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f32f0014410 con 0x7f330009c9e0 2024-09-22T03:46:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.113+0000 7f0f99ffb640 1 -- 172.21.15.3:0/2372675189 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f0fa0011710 con 0x7f0fa40a3e10 2024-09-22T03:46:59.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.113+0000 7f98f17fa640 1 -- 172.21.15.3:0/1237867726 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7f98e8009910 con 0x7f98f4134010 2024-09-22T03:46:59.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.113+0000 7f56277fe640 1 -- 172.21.15.3:0/4105127720 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5630005a60 con 0x7f5640131c10 2024-09-22T03:46:59.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.113+0000 7f3306791640 1 -- 172.21.15.3:0/1480892727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3300120c20 con 0x7f330009c9e0 2024-09-22T03:46:59.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.113+0000 7f5645791640 1 -- 172.21.15.3:0/4105127720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f564011e080 con 0x7f5640131c10 2024-09-22T03:46:59.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.113+0000 7f3306791640 1 -- 172.21.15.3:0/1480892727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f330011fd20 con 0x7f330009c9e0 2024-09-22T03:46:59.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.113+0000 7f56277fe640 1 -- 172.21.15.3:0/4105127720 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5630005bc0 con 0x7f5640131c10 2024-09-22T03:46:59.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.113+0000 7f56277fe640 1 -- 172.21.15.3:0/4105127720 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5630005a60 con 0x7f5640131c10 2024-09-22T03:46:59.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.114+0000 7f5645791640 1 -- 172.21.15.3:0/4105127720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f564011e750 con 0x7f5640131c10 2024-09-22T03:46:59.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.114+0000 7f32fcff9640 1 -- 172.21.15.3:0/1480892727 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f32f0009400 con 0x7f330009c9e0 2024-09-22T03:46:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.114+0000 7f32fcff9640 1 -- 172.21.15.3:0/1480892727 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f32f00146a0 con 0x7f330009c9e0 2024-09-22T03:46:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.115+0000 7f5645791640 1 -- 172.21.15.3:0/4105127720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5640132020 con 0x7f5640131c10 2024-09-22T03:46:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.115+0000 7f3306791640 1 -- 172.21.15.3:0/1480892727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e00050f0 con 0x7f330009c9e0 2024-09-22T03:46:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.124+0000 7f98f17fa640 1 -- 172.21.15.3:0/1237867726 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f98e800e1c0 con 0x7f98f4134010 2024-09-22T03:46:59.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.125+0000 7f56277fe640 1 -- 172.21.15.3:0/4105127720 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f56300070b0 con 0x7f5640131c10 2024-09-22T03:46:59.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.126+0000 7f56277fe640 1 --2- 172.21.15.3:0/4105127720 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f562c03eb00 0x7f562c041010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:59.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.126+0000 7f56277fe640 1 -- 172.21.15.3:0/4105127720 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7f563004f7f0 con 0x7f5640131c10 2024-09-22T03:46:59.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.126+0000 7f32fcff9640 1 -- 172.21.15.3:0/1480892727 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f32f00148a0 con 0x7f330009c9e0 2024-09-22T03:46:59.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.126+0000 7f563e7fc640 1 --2- 172.21.15.3:0/4105127720 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f562c03eb00 0x7f562c041010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:59.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.127+0000 7f32fcff9640 1 --2- 172.21.15.3:0/1480892727 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f32ec03ead0 0x7f32ec040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:46:59.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.127+0000 7f32fcff9640 1 -- 172.21.15.3:0/1480892727 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7f32f0019040 con 0x7f330009c9e0 2024-09-22T03:46:59.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.127+0000 7f32ff7fe640 1 --2- 172.21.15.3:0/1480892727 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f32ec03ead0 0x7f32ec040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:46:59.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.127+0000 7f563e7fc640 1 --2- 172.21.15.3:0/4105127720 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f562c03eb00 0x7f562c041010 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:59.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.127+0000 7f32ff7fe640 1 --2- 172.21.15.3:0/1480892727 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f32ec03ead0 0x7f32ec040fe0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:46:59.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.137+0000 7f56277fe640 1 -- 172.21.15.3:0/4105127720 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f563000abe0 con 0x7f5640131c10 2024-09-22T03:46:59.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.147+0000 7f32fcff9640 1 -- 172.21.15.3:0/1480892727 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f32f000d1c0 con 0x7f330009c9e0 2024-09-22T03:46:59.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.312+0000 7f0fab85a640 1 -- 172.21.15.3:0/2372675189 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0fa40a56d0 con 0x7f0fa40a3e10 2024-09-22T03:46:59.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.313+0000 7f0f99ffb640 1 -- 172.21.15.3:0/2372675189 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f0fa00118b0 con 0x7f0fa40a3e10 2024-09-22T03:46:59.314 INFO:teuthology.orchestra.run.smithi003.stdout:38654705668 2024-09-22T03:46:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.316+0000 7f0fab85a640 1 -- 172.21.15.3:0/2372675189 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f0f8803eab0 msgr2=0x7f0f88040fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.316+0000 7f0fab85a640 1 --2- 172.21.15.3:0/2372675189 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f0f8803eab0 0x7f0f88040fc0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.316+0000 7f0fab85a640 1 -- 172.21.15.3:0/2372675189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa40a3e10 msgr2=0x7f0fa4123df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.316+0000 7f0fab85a640 1 --2- 172.21.15.3:0/2372675189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa40a3e10 0x7f0fa4123df0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.316+0000 7f0fab85a640 1 -- 172.21.15.3:0/2372675189 shutdown_connections 2024-09-22T03:46:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.316+0000 7f0fab85a640 1 --2- 172.21.15.3:0/2372675189 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f0f8803eab0 0x7f0f88040fc0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.316+0000 7f0fab85a640 1 --2- 172.21.15.3:0/2372675189 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fa40a3e10 0x7f0fa4123df0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.316+0000 7f0fab85a640 1 -- 172.21.15.3:0/2372675189 >> 172.21.15.3:0/2372675189 conn(0x7f0fa40a74f0 msgr2=0x7f0fa40713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.316+0000 7f0fab85a640 1 -- 172.21.15.3:0/2372675189 shutdown_connections 2024-09-22T03:46:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.316+0000 7f0fab85a640 1 -- 172.21.15.3:0/2372675189 wait complete. 2024-09-22T03:46:59.329 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705672 got 38654705668 for osd.0 2024-09-22T03:46:59.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.328+0000 7f5645791640 1 -- 172.21.15.3:0/4105127720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f56400773f0 con 0x7f5640131c10 2024-09-22T03:46:59.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.329+0000 7f56277fe640 1 -- 172.21.15.3:0/4105127720 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f5630010e60 con 0x7f5640131c10 2024-09-22T03:46:59.330 INFO:teuthology.orchestra.run.smithi003.stdout:38654705668 2024-09-22T03:46:59.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.332+0000 7f5645791640 1 -- 172.21.15.3:0/4105127720 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f562c03eb00 msgr2=0x7f562c041010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:59.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.332+0000 7f5645791640 1 --2- 172.21.15.3:0/4105127720 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f562c03eb00 0x7f562c041010 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.332+0000 7f5645791640 1 -- 172.21.15.3:0/4105127720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5640131c10 msgr2=0x7f564011ccd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:59.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.332+0000 7f5645791640 1 --2- 172.21.15.3:0/4105127720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5640131c10 0x7f564011ccd0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.332+0000 7f5645791640 1 -- 172.21.15.3:0/4105127720 shutdown_connections 2024-09-22T03:46:59.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.332+0000 7f5645791640 1 --2- 172.21.15.3:0/4105127720 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f562c03eb00 0x7f562c041010 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.332+0000 7f5645791640 1 --2- 172.21.15.3:0/4105127720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5640131c10 0x7f564011ccd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.332+0000 7f5645791640 1 -- 172.21.15.3:0/4105127720 >> 172.21.15.3:0/4105127720 conn(0x7f564006e770 msgr2=0x7f56401338f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:59.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.332+0000 7f5645791640 1 -- 172.21.15.3:0/4105127720 shutdown_connections 2024-09-22T03:46:59.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.332+0000 7f5645791640 1 -- 172.21.15.3:0/4105127720 wait complete. 2024-09-22T03:46:59.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705672 got 38654705668 for osd.3 2024-09-22T03:46:59.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.350+0000 7f98fab11640 1 -- 172.21.15.3:0/1237867726 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f98f4124460 con 0x7f98f4134010 2024-09-22T03:46:59.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.350+0000 7f98f17fa640 1 -- 172.21.15.3:0/1237867726 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f98e8009570 con 0x7f98f4134010 2024-09-22T03:46:59.352 INFO:teuthology.orchestra.run.smithi003.stdout:38654705668 2024-09-22T03:46:59.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.352+0000 7f98fab11640 1 -- 172.21.15.3:0/1237867726 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f98d803ead0 msgr2=0x7f98d8040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:59.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.352+0000 7f98fab11640 1 --2- 172.21.15.3:0/1237867726 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f98d803ead0 0x7f98d8040fe0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.352+0000 7f98fab11640 1 -- 172.21.15.3:0/1237867726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f4134010 msgr2=0x7f98f4121080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:59.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.352+0000 7f98fab11640 1 --2- 172.21.15.3:0/1237867726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f4134010 0x7f98f4121080 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.352+0000 7f98fab11640 1 -- 172.21.15.3:0/1237867726 shutdown_connections 2024-09-22T03:46:59.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.352+0000 7f98fab11640 1 --2- 172.21.15.3:0/1237867726 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f98d803ead0 0x7f98d8040fe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.352+0000 7f98fab11640 1 --2- 172.21.15.3:0/1237867726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98f4134010 0x7f98f4121080 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.352+0000 7f98fab11640 1 -- 172.21.15.3:0/1237867726 >> 172.21.15.3:0/1237867726 conn(0x7f98f40a43e0 msgr2=0x7f98f4134aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:59.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.353+0000 7f98fab11640 1 -- 172.21.15.3:0/1237867726 shutdown_connections 2024-09-22T03:46:59.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.353+0000 7f98fab11640 1 -- 172.21.15.3:0/1237867726 wait complete. 2024-09-22T03:46:59.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.359+0000 7f3306791640 1 -- 172.21.15.3:0/1480892727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f32e0005670 con 0x7f330009c9e0 2024-09-22T03:46:59.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.360+0000 7f32fcff9640 1 -- 172.21.15.3:0/1480892727 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f32f0018190 con 0x7f330009c9e0 2024-09-22T03:46:59.361 INFO:teuthology.orchestra.run.smithi003.stdout:38654705668 2024-09-22T03:46:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.362+0000 7f3306791640 1 -- 172.21.15.3:0/1480892727 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f32ec03ead0 msgr2=0x7f32ec040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.362+0000 7f3306791640 1 --2- 172.21.15.3:0/1480892727 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f32ec03ead0 0x7f32ec040fe0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.362+0000 7f3306791640 1 -- 172.21.15.3:0/1480892727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f330009c9e0 msgr2=0x7f330011e970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:46:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.362+0000 7f3306791640 1 --2- 172.21.15.3:0/1480892727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f330009c9e0 0x7f330011e970 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.362+0000 7f3306791640 1 -- 172.21.15.3:0/1480892727 shutdown_connections 2024-09-22T03:46:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.362+0000 7f3306791640 1 --2- 172.21.15.3:0/1480892727 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f32ec03ead0 0x7f32ec040fe0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.362+0000 7f3306791640 1 --2- 172.21.15.3:0/1480892727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f330009c9e0 0x7f330011e970 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:46:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.362+0000 7f3306791640 1 -- 172.21.15.3:0/1480892727 >> 172.21.15.3:0/1480892727 conn(0x7f33000a43e0 msgr2=0x7f3300071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:46:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.362+0000 7f3306791640 1 -- 172.21.15.3:0/1480892727 shutdown_connections 2024-09-22T03:46:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:46:59.362+0000 7f3306791640 1 -- 172.21.15.3:0/1480892727 wait complete. 2024-09-22T03:46:59.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705672 got 38654705668 for osd.2 2024-09-22T03:46:59.374 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705672 got 38654705668 for osd.1 2024-09-22T03:47:00.330 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T03:47:00.345 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T03:47:00.366 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T03:47:00.375 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T03:47:00.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.455+0000 7f5c32191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:47:00.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.465+0000 7f5c32191640 1 Processor -- start 2024-09-22T03:47:00.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.465+0000 7f5c32191640 1 -- start start 2024-09-22T03:47:00.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.466+0000 7f5c32191640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c2c0a4930 0x7f5c2c13b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:00.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.466+0000 7f5c32191640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c2c06ec50 con 0x7f5c2c0a3e10 2024-09-22T03:47:00.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.466+0000 7f5c32191640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f5c2c0a6130 con 0x7f5c2c0a4930 2024-09-22T03:47:00.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.467+0000 7f5c2affd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c2c0a4930 0x7f5c2c13b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:00.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.467+0000 7f5c2affd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5c2c0a4930 0x7f5c2c13b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58974/0 (socket says 172.21.15.3:58974) 2024-09-22T03:47:00.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.467+0000 7f5c2affd640 1 -- 172.21.15.3:0/562403831 learned_addr learned my addr 172.21.15.3:0/562403831 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:47:00.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.467+0000 7f5c2affd640 1 -- 172.21.15.3:0/562403831 >> v1:172.21.15.3:6789/0 conn(0x7f5c2c0a3e10 legacy=0x7f5c2c0a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:00.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.467+0000 7f5c2affd640 1 -- 172.21.15.3:0/562403831 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2c0a5110 con 0x7f5c2c0a4930 2024-09-22T03:47:00.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.467+0000 7f5c2affd640 1 --2- 172.21.15.3:0/562403831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0a4930 0x7f5c2c13b2d0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf6d2bccd3159a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:00.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.467+0000 7f5c2a7fc640 1 -- 172.21.15.3:0/562403831 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5c20009e50 con 0x7f5c2c0a4930 2024-09-22T03:47:00.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.467+0000 7f5c2a7fc640 1 -- 172.21.15.3:0/562403831 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5c2000e230 con 0x7f5c2c0a4930 2024-09-22T03:47:00.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.467+0000 7f5c2a7fc640 1 -- 172.21.15.3:0/562403831 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5c2000e4d0 con 0x7f5c2c0a4930 2024-09-22T03:47:00.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.468+0000 7f5c32191640 1 -- 172.21.15.3:0/562403831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0a4930 msgr2=0x7f5c2c13b2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:00.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.468+0000 7f5c32191640 1 --2- 172.21.15.3:0/562403831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0a4930 0x7f5c2c13b2d0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.468+0000 7f5c32191640 1 -- 172.21.15.3:0/562403831 shutdown_connections 2024-09-22T03:47:00.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.468+0000 7f5c32191640 1 --2- 172.21.15.3:0/562403831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0a4930 0x7f5c2c13b2d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.468+0000 7f5c32191640 1 -- 172.21.15.3:0/562403831 >> 172.21.15.3:0/562403831 conn(0x7f5c2c0a74f0 msgr2=0x7f5c2c0a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:47:00.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.469+0000 7f5c32191640 1 -- 172.21.15.3:0/562403831 shutdown_connections 2024-09-22T03:47:00.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.469+0000 7f5c32191640 1 -- 172.21.15.3:0/562403831 wait complete. 2024-09-22T03:47:00.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.469+0000 7f5c32191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:47:00.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.471+0000 7f5c32191640 1 Processor -- start 2024-09-22T03:47:00.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.471+0000 7f5c32191640 1 -- start start 2024-09-22T03:47:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.472+0000 7f5c32191640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0a3e10 0x7f5c2c123920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.472+0000 7f5c32191640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c1253a0 con 0x7f5c2c0a3e10 2024-09-22T03:47:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.473+0000 7f5c2b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0a3e10 0x7f5c2c123920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.473+0000 7f5c2b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0a3e10 0x7f5c2c123920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58982/0 (socket says 172.21.15.3:58982) 2024-09-22T03:47:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.473+0000 7f5c2b7fe640 1 -- 172.21.15.3:0/891775606 learned_addr learned my addr 172.21.15.3:0/891775606 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:47:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.473+0000 7f5c2b7fe640 1 -- 172.21.15.3:0/891775606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2c124b60 con 0x7f5c2c0a3e10 2024-09-22T03:47:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.473+0000 7f5c2b7fe640 1 --2- 172.21.15.3:0/891775606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0a3e10 0x7f5c2c123920 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.474+0000 7f5c13fff640 1 -- 172.21.15.3:0/891775606 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5c1c00c850 con 0x7f5c2c0a3e10 2024-09-22T03:47:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.474+0000 7f5c13fff640 1 -- 172.21.15.3:0/891775606 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5c1c00c9b0 con 0x7f5c2c0a3e10 2024-09-22T03:47:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.474+0000 7f5c32191640 1 -- 172.21.15.3:0/891775606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2c125c50 con 0x7f5c2c0a3e10 2024-09-22T03:47:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.474+0000 7f5c13fff640 1 -- 172.21.15.3:0/891775606 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5c1c00cc50 con 0x7f5c2c0a3e10 2024-09-22T03:47:00.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.474+0000 7f5c32191640 1 -- 172.21.15.3:0/891775606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c2c1263d0 con 0x7f5c2c0a3e10 2024-09-22T03:47:00.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.475+0000 7f5c32191640 1 -- 172.21.15.3:0/891775606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c2c1277f0 con 0x7f5c2c0a3e10 2024-09-22T03:47:00.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.476+0000 7f5c13fff640 1 -- 172.21.15.3:0/891775606 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f5c1c011310 con 0x7f5c2c0a3e10 2024-09-22T03:47:00.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.476+0000 7f5c13fff640 1 --2- 172.21.15.3:0/891775606 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5c1803ead0 0x7f5c18040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:00.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.476+0000 7f5c13fff640 1 -- 172.21.15.3:0/891775606 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7f5c1c04ffa0 con 0x7f5c2c0a3e10 2024-09-22T03:47:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.477+0000 7f5c2affd640 1 --2- 172.21.15.3:0/891775606 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5c1803ead0 0x7f5c18040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.477+0000 7f5c2affd640 1 --2- 172.21.15.3:0/891775606 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5c1803ead0 0x7f5c18040fe0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:00.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.481+0000 7f5c13fff640 1 -- 172.21.15.3:0/891775606 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f5c1c00d150 con 0x7f5c2c0a3e10 2024-09-22T03:47:00.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.495+0000 7fe703a23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:47:00.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.495+0000 7f3e05991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:47:00.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.502+0000 7fe703a23640 1 Processor -- start 2024-09-22T03:47:00.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.502+0000 7fe703a23640 1 -- start start 2024-09-22T03:47:00.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.502+0000 7f3e05991640 1 Processor -- start 2024-09-22T03:47:00.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.502+0000 7f3e05991640 1 -- start start 2024-09-22T03:47:00.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.504+0000 7fe703a23640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe6fc0a3e10 0x7fe6fc0a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:00.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.504+0000 7fe703a23640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6fc06ec50 con 0x7fe6fc0a4760 2024-09-22T03:47:00.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.505+0000 7fe703a23640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe6fc0a6130 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.505+0000 7f3e05991640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3e000a3e10 0x7f3e000a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:00.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.505+0000 7f3e05991640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e0006ec50 con 0x7f3e000a4760 2024-09-22T03:47:00.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.505+0000 7fe700fbe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe6fc0a4760 0x7fe6fc135e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33222/0 (socket says 172.21.15.3:33222) 2024-09-22T03:47:00.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.505+0000 7fe700fbe640 1 -- 172.21.15.3:0/1431438148 learned_addr learned my addr 172.21.15.3:0/1431438148 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T03:47:00.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.505+0000 7fe7017bf640 1 --2- 172.21.15.3:0/1431438148 >> v2:172.21.15.3:3300/0 conn(0x7fe6fc0a3e10 0x7fe6fc0a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:00.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.505+0000 7f3e05991640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3e000a6130 con 0x7f3e000a3e10 2024-09-22T03:47:00.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.505+0000 7f3dfeffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3e000a3e10 0x7f3e000a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:00.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.505+0000 7f3dfeffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3e000a3e10 0x7f3e000a4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59002/0 (socket says 172.21.15.3:59002) 2024-09-22T03:47:00.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.505+0000 7f3dfeffd640 1 -- 172.21.15.3:0/27192552 learned_addr learned my addr 172.21.15.3:0/27192552 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:47:00.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.505+0000 7fe6f3fff640 1 -- 172.21.15.3:0/1431438148 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2569081623 0 0) 0x7fe6fc06ec50 con 0x7fe6fc0a4760 2024-09-22T03:47:00.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.505+0000 7fe6f3fff640 1 -- 172.21.15.3:0/1431438148 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6e8003580 con 0x7fe6fc0a4760 2024-09-22T03:47:00.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.505+0000 7f3dfeffd640 1 -- 172.21.15.3:0/27192552 >> v1:172.21.15.3:6789/0 conn(0x7f3e000a4760 legacy=0x7f3e00135e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-22T03:47:00.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.505+0000 7f3dfeffd640 1 -- 172.21.15.3:0/27192552 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e000a5110 con 0x7f3e000a3e10 2024-09-22T03:47:00.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.506+0000 7f3dfeffd640 1 --2- 172.21.15.3:0/27192552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e000a3e10 0x7f3e000a4220 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7cfbf41d1262c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:00.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.505+0000 7fe7017bf640 1 -- 172.21.15.3:0/1431438148 >> v1:172.21.15.3:6789/0 conn(0x7fe6fc0a4760 legacy=0x7fe6fc135e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:00.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.505+0000 7fe7017bf640 1 -- 172.21.15.3:0/1431438148 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc0a5110 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.506+0000 7fe7017bf640 1 --2- 172.21.15.3:0/1431438148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0a3e10 0x7fe6fc0a4220 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f3d40f940201c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:00.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.506+0000 7fe6f3fff640 1 -- 172.21.15.3:0/1431438148 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe6f8009d90 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.506+0000 7f3dfdffb640 1 -- 172.21.15.3:0/27192552 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3df0009db0 con 0x7f3e000a3e10 2024-09-22T03:47:00.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.506+0000 7fe6f3fff640 1 -- 172.21.15.3:0/1431438148 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe6f800e230 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.500+0000 7f4daef91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:47:00.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.507+0000 7f3dfdffb640 1 -- 172.21.15.3:0/27192552 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3df000e230 con 0x7f3e000a3e10 2024-09-22T03:47:00.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.507+0000 7f3dfdffb640 1 -- 172.21.15.3:0/27192552 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3df0009db0 con 0x7f3e000a3e10 2024-09-22T03:47:00.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.507+0000 7fe6f3fff640 1 -- 172.21.15.3:0/1431438148 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe6f800e4d0 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.507+0000 7fe703a23640 1 -- 172.21.15.3:0/1431438148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0a3e10 msgr2=0x7fe6fc0a4220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:00.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.507+0000 7fe703a23640 1 --2- 172.21.15.3:0/1431438148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0a3e10 0x7fe6fc0a4220 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.507+0000 7fe703a23640 1 -- 172.21.15.3:0/1431438148 shutdown_connections 2024-09-22T03:47:00.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.507+0000 7fe703a23640 1 --2- 172.21.15.3:0/1431438148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0a3e10 0x7fe6fc0a4220 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.507+0000 7fe703a23640 1 -- 172.21.15.3:0/1431438148 >> 172.21.15.3:0/1431438148 conn(0x7fe6fc0a74f0 msgr2=0x7fe6fc0a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:47:00.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.508+0000 7fe703a23640 1 -- 172.21.15.3:0/1431438148 shutdown_connections 2024-09-22T03:47:00.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.508+0000 7fe703a23640 1 -- 172.21.15.3:0/1431438148 wait complete. 2024-09-22T03:47:00.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.508+0000 7f3e05991640 1 -- 172.21.15.3:0/27192552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e000a3e10 msgr2=0x7f3e000a4220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:00.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.508+0000 7f3e05991640 1 --2- 172.21.15.3:0/27192552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e000a3e10 0x7f3e000a4220 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.508+0000 7fe703a23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:47:00.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.508+0000 7f3e05991640 1 -- 172.21.15.3:0/27192552 shutdown_connections 2024-09-22T03:47:00.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.508+0000 7f4daef91640 1 Processor -- start 2024-09-22T03:47:00.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.508+0000 7f3e05991640 1 --2- 172.21.15.3:0/27192552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e000a3e10 0x7f3e000a4220 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.508+0000 7f3e05991640 1 -- 172.21.15.3:0/27192552 >> 172.21.15.3:0/27192552 conn(0x7f3e000a74f0 msgr2=0x7f3e000a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:47:00.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.508+0000 7f3e05991640 1 -- 172.21.15.3:0/27192552 shutdown_connections 2024-09-22T03:47:00.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.508+0000 7f4daef91640 1 -- start start 2024-09-22T03:47:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.508+0000 7f3e05991640 1 -- 172.21.15.3:0/27192552 wait complete. 2024-09-22T03:47:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.508+0000 7f3e05991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:47:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.510+0000 7f3e05991640 1 Processor -- start 2024-09-22T03:47:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.510+0000 7f3e05991640 1 -- start start 2024-09-22T03:47:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.511+0000 7f4daef91640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4da80a3e10 0x7f4da80a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.511+0000 7fe703a23640 1 Processor -- start 2024-09-22T03:47:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.511+0000 7fe703a23640 1 -- start start 2024-09-22T03:47:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.511+0000 7f4daef91640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4da806ec50 con 0x7f4da80a4760 2024-09-22T03:47:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.511+0000 7f4daef91640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4da80a6130 con 0x7f4da80a3e10 2024-09-22T03:47:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.511+0000 7f4dacd2d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4da80a3e10 0x7f4da80a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.511+0000 7f4da7fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4da80a4760 0x7f4da8135e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33242/0 (socket says 172.21.15.3:33242) 2024-09-22T03:47:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.511+0000 7f4da7fff640 1 -- 172.21.15.3:0/3967908123 learned_addr learned my addr 172.21.15.3:0/3967908123 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T03:47:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.511+0000 7f4dacd2d640 1 -- 172.21.15.3:0/3967908123 >> v1:172.21.15.3:6789/0 conn(0x7f4da80a4760 legacy=0x7f4da8135e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.511+0000 7f4dacd2d640 1 -- 172.21.15.3:0/3967908123 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da80a5110 con 0x7f4da80a3e10 2024-09-22T03:47:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.512+0000 7f4dacd2d640 1 --2- 172.21.15.3:0/3967908123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4da80a3e10 0x7f4da80a4220 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b6b9101b8671aa server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.512+0000 7f4da77fe640 1 -- 172.21.15.3:0/3967908123 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4d9c009d90 con 0x7f4da80a3e10 2024-09-22T03:47:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.512+0000 7f4da77fe640 1 -- 172.21.15.3:0/3967908123 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4d9c00e230 con 0x7f4da80a3e10 2024-09-22T03:47:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.512+0000 7f3e05991640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e000a3e10 0x7f3e00124220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.512+0000 7fe703a23640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0a3e10 0x7fe6fc123df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:00.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.512+0000 7f3e05991640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3e00123840 con 0x7f3e000a3e10 2024-09-22T03:47:00.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.513+0000 7fe7017bf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0a3e10 0x7fe6fc123df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:00.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.513+0000 7fe7017bf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0a3e10 0x7fe6fc123df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59040/0 (socket says 172.21.15.3:59040) 2024-09-22T03:47:00.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.513+0000 7fe7017bf640 1 -- 172.21.15.3:0/1048336501 learned_addr learned my addr 172.21.15.3:0/1048336501 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:47:00.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.513+0000 7fe703a23640 1 -- 172.21.15.3:0/1048336501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc126070 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.512+0000 7f3dfeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e000a3e10 0x7f3e00124220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:00.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.513+0000 7f3dfeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e000a3e10 0x7f3e00124220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59026/0 (socket says 172.21.15.3:59026) 2024-09-22T03:47:00.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.513+0000 7f4da77fe640 1 -- 172.21.15.3:0/3967908123 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4d9c00e4d0 con 0x7f4da80a3e10 2024-09-22T03:47:00.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.513+0000 7f3dfeffd640 1 -- 172.21.15.3:0/413167288 learned_addr learned my addr 172.21.15.3:0/413167288 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:47:00.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.513+0000 7fe7017bf640 1 -- 172.21.15.3:0/1048336501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc1251c0 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.513+0000 7fe7017bf640 1 --2- 172.21.15.3:0/1048336501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0a3e10 0x7fe6fc123df0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:00.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.513+0000 7f4daef91640 1 -- 172.21.15.3:0/3967908123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4da80a3e10 msgr2=0x7f4da80a4220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:00.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.513+0000 7f4daef91640 1 --2- 172.21.15.3:0/3967908123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4da80a3e10 0x7f4da80a4220 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.513+0000 7f4daef91640 1 -- 172.21.15.3:0/3967908123 shutdown_connections 2024-09-22T03:47:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.513+0000 7f4daef91640 1 --2- 172.21.15.3:0/3967908123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4da80a3e10 0x7f4da80a4220 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.513+0000 7f4daef91640 1 -- 172.21.15.3:0/3967908123 >> 172.21.15.3:0/3967908123 conn(0x7f4da80a74f0 msgr2=0x7f4da80a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:47:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.513+0000 7f4daef91640 1 -- 172.21.15.3:0/3967908123 shutdown_connections 2024-09-22T03:47:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.514+0000 7f4daef91640 1 -- 172.21.15.3:0/3967908123 wait complete. 2024-09-22T03:47:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.514+0000 7f4daef91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:47:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.514+0000 7fe6f1ffb640 1 -- 172.21.15.3:0/1048336501 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe6f800e4d0 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.514+0000 7f3dfeffd640 1 -- 172.21.15.3:0/413167288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e00123dd0 con 0x7f3e000a3e10 2024-09-22T03:47:00.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.514+0000 7fe6f1ffb640 1 -- 172.21.15.3:0/1048336501 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe6f8013610 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.514+0000 7fe703a23640 1 -- 172.21.15.3:0/1048336501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6fc123780 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.514+0000 7fe6f1ffb640 1 -- 172.21.15.3:0/1048336501 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe6f80138b0 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.514+0000 7f3dfeffd640 1 --2- 172.21.15.3:0/413167288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e000a3e10 0x7f3e00124220 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:00.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.514+0000 7fe703a23640 1 -- 172.21.15.3:0/1048336501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6fc123a50 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.515+0000 7f3de7fff640 1 -- 172.21.15.3:0/413167288 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3df000ecf0 con 0x7f3e000a3e10 2024-09-22T03:47:00.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.515+0000 7f3de7fff640 1 -- 172.21.15.3:0/413167288 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3df000ee50 con 0x7f3e000a3e10 2024-09-22T03:47:00.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.515+0000 7f3de7fff640 1 -- 172.21.15.3:0/413167288 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3df0015550 con 0x7f3e000a3e10 2024-09-22T03:47:00.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.515+0000 7f3e05991640 1 -- 172.21.15.3:0/413167288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e00123a30 con 0x7f3e000a3e10 2024-09-22T03:47:00.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.515+0000 7f3e05991640 1 -- 172.21.15.3:0/413167288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e00127310 con 0x7f3e000a3e10 2024-09-22T03:47:00.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.516+0000 7fe6f1ffb640 1 -- 172.21.15.3:0/1048336501 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fe6f8013ab0 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.516+0000 7fe703a23640 1 -- 172.21.15.3:0/1048336501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6fc125ae0 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.516+0000 7f4daef91640 1 Processor -- start 2024-09-22T03:47:00.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.516+0000 7fe6f1ffb640 1 --2- 172.21.15.3:0/1048336501 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe6e003eab0 0x7fe6e0040fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:00.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.516+0000 7f4daef91640 1 -- start start 2024-09-22T03:47:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.516+0000 7f3e05991640 1 -- 172.21.15.3:0/413167288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e000a56d0 con 0x7f3e000a3e10 2024-09-22T03:47:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.516+0000 7fe700fbe640 1 --2- 172.21.15.3:0/1048336501 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe6e003eab0 0x7fe6e0040fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.517+0000 7fe700fbe640 1 --2- 172.21.15.3:0/1048336501 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe6e003eab0 0x7fe6e0040fc0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:00.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.517+0000 7f3de7fff640 1 -- 172.21.15.3:0/413167288 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f3df00156b0 con 0x7f3e000a3e10 2024-09-22T03:47:00.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.517+0000 7fe6f1ffb640 1 -- 172.21.15.3:0/1048336501 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7fe6f8018040 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.517+0000 7f3de7fff640 1 --2- 172.21.15.3:0/413167288 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f3dec03ead0 0x7f3dec040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:00.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.517+0000 7f3de7fff640 1 -- 172.21.15.3:0/413167288 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7f3df0050f10 con 0x7f3e000a3e10 2024-09-22T03:47:00.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.518+0000 7f3dfe7fc640 1 --2- 172.21.15.3:0/413167288 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f3dec03ead0 0x7f3dec040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.518+0000 7f4daef91640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4da80a3e10 0x7f4da8123be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.518+0000 7f4daef91640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4da81278e0 con 0x7f4da80a3e10 2024-09-22T03:47:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.518+0000 7f4dacd2d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4da80a3e10 0x7f4da8123be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.518+0000 7f4dacd2d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4da80a3e10 0x7f4da8123be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59056/0 (socket says 172.21.15.3:59056) 2024-09-22T03:47:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.518+0000 7f4dacd2d640 1 -- 172.21.15.3:0/1401105671 learned_addr learned my addr 172.21.15.3:0/1401105671 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:47:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.518+0000 7f3dfe7fc640 1 --2- 172.21.15.3:0/413167288 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f3dec03ead0 0x7f3dec040fe0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:00.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.518+0000 7f4dacd2d640 1 -- 172.21.15.3:0/1401105671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da8125620 con 0x7f4da80a3e10 2024-09-22T03:47:00.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.518+0000 7f4dacd2d640 1 --2- 172.21.15.3:0/1401105671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4da80a3e10 0x7f4da8123be0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:00.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.528+0000 7f3de7fff640 1 -- 172.21.15.3:0/413167288 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f3df0014c60 con 0x7f3e000a3e10 2024-09-22T03:47:00.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.538+0000 7f4da57fa640 1 -- 172.21.15.3:0/1401105671 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4d9c014750 con 0x7f4da80a3e10 2024-09-22T03:47:00.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.539+0000 7fe6f1ffb640 1 -- 172.21.15.3:0/1048336501 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7fe6f8011380 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.539+0000 7f4daef91640 1 -- 172.21.15.3:0/1401105671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da8124d70 con 0x7f4da80a3e10 2024-09-22T03:47:00.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.539+0000 7f4daef91640 1 -- 172.21.15.3:0/1401105671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da8126f00 con 0x7f4da80a3e10 2024-09-22T03:47:00.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.539+0000 7f4da57fa640 1 -- 172.21.15.3:0/1401105671 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4d9c0148b0 con 0x7f4da80a3e10 2024-09-22T03:47:00.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.539+0000 7f4da57fa640 1 -- 172.21.15.3:0/1401105671 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4d9c014a20 con 0x7f4da80a3e10 2024-09-22T03:47:00.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.540+0000 7f4daef91640 1 -- 172.21.15.3:0/1401105671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d880050f0 con 0x7f4da80a3e10 2024-09-22T03:47:00.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.540+0000 7f4da57fa640 1 -- 172.21.15.3:0/1401105671 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f4d9c013400 con 0x7f4da80a3e10 2024-09-22T03:47:00.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.541+0000 7f4da57fa640 1 --2- 172.21.15.3:0/1401105671 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4d8c03ea80 0x7f4d8c040f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:00.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.541+0000 7f4da57fa640 1 -- 172.21.15.3:0/1401105671 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7f4d9c019040 con 0x7f4da80a3e10 2024-09-22T03:47:00.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.541+0000 7f4da7fff640 1 --2- 172.21.15.3:0/1401105671 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4d8c03ea80 0x7f4d8c040f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:00.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.542+0000 7f4da7fff640 1 --2- 172.21.15.3:0/1401105671 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4d8c03ea80 0x7f4d8c040f90 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:00.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.551+0000 7f4da57fa640 1 -- 172.21.15.3:0/1401105671 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f4d9c018970 con 0x7f4da80a3e10 2024-09-22T03:47:00.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.700+0000 7f5c32191640 1 -- 172.21.15.3:0/891775606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5c2c06e050 con 0x7f5c2c0a3e10 2024-09-22T03:47:00.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.701+0000 7f5c13fff640 1 -- 172.21.15.3:0/891775606 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f5c1c00a9b0 con 0x7f5c2c0a3e10 2024-09-22T03:47:00.703 INFO:teuthology.orchestra.run.smithi003.stdout:38654705672 2024-09-22T03:47:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.703+0000 7f5c32191640 1 -- 172.21.15.3:0/891775606 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5c1803ead0 msgr2=0x7f5c18040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.703+0000 7f5c32191640 1 --2- 172.21.15.3:0/891775606 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5c1803ead0 0x7f5c18040fe0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.703+0000 7f5c32191640 1 -- 172.21.15.3:0/891775606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0a3e10 msgr2=0x7f5c2c123920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.703+0000 7f5c32191640 1 --2- 172.21.15.3:0/891775606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0a3e10 0x7f5c2c123920 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.703+0000 7f5c32191640 1 -- 172.21.15.3:0/891775606 shutdown_connections 2024-09-22T03:47:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.703+0000 7f5c32191640 1 --2- 172.21.15.3:0/891775606 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5c1803ead0 0x7f5c18040fe0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.703+0000 7f5c32191640 1 --2- 172.21.15.3:0/891775606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c2c0a3e10 0x7f5c2c123920 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.703+0000 7f5c32191640 1 -- 172.21.15.3:0/891775606 >> 172.21.15.3:0/891775606 conn(0x7f5c2c0a74f0 msgr2=0x7f5c2c071470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:47:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.703+0000 7f5c32191640 1 -- 172.21.15.3:0/891775606 shutdown_connections 2024-09-22T03:47:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.703+0000 7f5c32191640 1 -- 172.21.15.3:0/891775606 wait complete. 2024-09-22T03:47:00.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705672 got 38654705672 for osd.0 2024-09-22T03:47:00.716 DEBUG:teuthology.parallel:result is None 2024-09-22T03:47:00.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.739+0000 7f3e05991640 1 -- 172.21.15.3:0/413167288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3e00077360 con 0x7f3e000a3e10 2024-09-22T03:47:00.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.740+0000 7f3de7fff640 1 -- 172.21.15.3:0/413167288 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f3df0010310 con 0x7f3e000a3e10 2024-09-22T03:47:00.741 INFO:teuthology.orchestra.run.smithi003.stdout:38654705672 2024-09-22T03:47:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.742+0000 7f3e05991640 1 -- 172.21.15.3:0/413167288 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f3dec03ead0 msgr2=0x7f3dec040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.742+0000 7f3e05991640 1 --2- 172.21.15.3:0/413167288 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f3dec03ead0 0x7f3dec040fe0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.742+0000 7f3e05991640 1 -- 172.21.15.3:0/413167288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e000a3e10 msgr2=0x7f3e00124220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.742+0000 7f3e05991640 1 --2- 172.21.15.3:0/413167288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e000a3e10 0x7f3e00124220 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.742+0000 7f3e05991640 1 -- 172.21.15.3:0/413167288 shutdown_connections 2024-09-22T03:47:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.742+0000 7f3e05991640 1 --2- 172.21.15.3:0/413167288 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f3dec03ead0 0x7f3dec040fe0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.742+0000 7f3e05991640 1 --2- 172.21.15.3:0/413167288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e000a3e10 0x7f3e00124220 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.742+0000 7f3e05991640 1 -- 172.21.15.3:0/413167288 >> 172.21.15.3:0/413167288 conn(0x7f3e000a74f0 msgr2=0x7f3e001bcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:47:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.742+0000 7f3e05991640 1 -- 172.21.15.3:0/413167288 shutdown_connections 2024-09-22T03:47:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.742+0000 7f4daef91640 1 -- 172.21.15.3:0/1401105671 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4d880053a0 con 0x7f4da80a3e10 2024-09-22T03:47:00.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.742+0000 7f3e05991640 1 -- 172.21.15.3:0/413167288 wait complete. 2024-09-22T03:47:00.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.743+0000 7f4da57fa640 1 -- 172.21.15.3:0/1401105671 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7f4d9c018190 con 0x7f4da80a3e10 2024-09-22T03:47:00.745 INFO:teuthology.orchestra.run.smithi003.stdout:38654705672 2024-09-22T03:47:00.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.745+0000 7f4daef91640 1 -- 172.21.15.3:0/1401105671 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4d8c03ea80 msgr2=0x7f4d8c040f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:00.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.745+0000 7f4daef91640 1 --2- 172.21.15.3:0/1401105671 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4d8c03ea80 0x7f4d8c040f90 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.745+0000 7f4daef91640 1 -- 172.21.15.3:0/1401105671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4da80a3e10 msgr2=0x7f4da8123be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:00.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.745+0000 7f4daef91640 1 --2- 172.21.15.3:0/1401105671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4da80a3e10 0x7f4da8123be0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.745+0000 7f4daef91640 1 -- 172.21.15.3:0/1401105671 shutdown_connections 2024-09-22T03:47:00.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.745+0000 7f4daef91640 1 --2- 172.21.15.3:0/1401105671 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4d8c03ea80 0x7f4d8c040f90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.745+0000 7f4daef91640 1 --2- 172.21.15.3:0/1401105671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4da80a3e10 0x7f4da8123be0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.745+0000 7f4daef91640 1 -- 172.21.15.3:0/1401105671 >> 172.21.15.3:0/1401105671 conn(0x7f4da80a74f0 msgr2=0x7f4da81bbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:47:00.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.745+0000 7f4daef91640 1 -- 172.21.15.3:0/1401105671 shutdown_connections 2024-09-22T03:47:00.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.746+0000 7f4daef91640 1 -- 172.21.15.3:0/1401105671 wait complete. 2024-09-22T03:47:00.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.747+0000 7fe703a23640 1 -- 172.21.15.3:0/1048336501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe6fc0a56d0 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.748+0000 7fe6f1ffb640 1 -- 172.21.15.3:0/1048336501 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (crc 0 0 0) 0x7fe6f8017a20 con 0x7fe6fc0a3e10 2024-09-22T03:47:00.749 INFO:teuthology.orchestra.run.smithi003.stdout:38654705672 2024-09-22T03:47:00.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.750+0000 7fe703a23640 1 -- 172.21.15.3:0/1048336501 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe6e003eab0 msgr2=0x7fe6e0040fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:00.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.750+0000 7fe703a23640 1 --2- 172.21.15.3:0/1048336501 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe6e003eab0 0x7fe6e0040fc0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.750+0000 7fe703a23640 1 -- 172.21.15.3:0/1048336501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0a3e10 msgr2=0x7fe6fc123df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:00.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.750+0000 7fe703a23640 1 --2- 172.21.15.3:0/1048336501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0a3e10 0x7fe6fc123df0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.750+0000 7fe703a23640 1 -- 172.21.15.3:0/1048336501 shutdown_connections 2024-09-22T03:47:00.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.750+0000 7fe703a23640 1 --2- 172.21.15.3:0/1048336501 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe6e003eab0 0x7fe6e0040fc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.750+0000 7fe703a23640 1 --2- 172.21.15.3:0/1048336501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0a3e10 0x7fe6fc123df0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.750+0000 7fe703a23640 1 -- 172.21.15.3:0/1048336501 >> 172.21.15.3:0/1048336501 conn(0x7fe6fc0a74f0 msgr2=0x7fe6fc0713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:47:00.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.750+0000 7fe703a23640 1 -- 172.21.15.3:0/1048336501 shutdown_connections 2024-09-22T03:47:00.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.750+0000 7fe703a23640 1 -- 172.21.15.3:0/1048336501 wait complete. 2024-09-22T03:47:00.756 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705672 got 38654705672 for osd.2 2024-09-22T03:47:00.756 DEBUG:teuthology.parallel:result is None 2024-09-22T03:47:00.760 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705672 got 38654705672 for osd.1 2024-09-22T03:47:00.760 DEBUG:teuthology.parallel:result is None 2024-09-22T03:47:00.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705672 got 38654705672 for osd.3 2024-09-22T03:47:00.763 DEBUG:teuthology.parallel:result is None 2024-09-22T03:47:00.763 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-22T03:47:00.764 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:47:00.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.911+0000 7fac81551640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:47:00.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.916+0000 7fac81551640 1 Processor -- start 2024-09-22T03:47:00.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.916+0000 7fac81551640 1 -- start start 2024-09-22T03:47:00.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.918+0000 7fac81551640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac7c09d0f0 0x7fac7c138ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:00.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.918+0000 7fac81551640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac7c074a70 con 0x7fac7c09e6c0 2024-09-22T03:47:00.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.918+0000 7fac81551640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fac7c0a61f0 con 0x7fac7c09d0f0 2024-09-22T03:47:00.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.918+0000 7fac7affd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fac7c09e6c0 0x7fac7c09c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33256/0 (socket says 172.21.15.3:33256) 2024-09-22T03:47:00.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.918+0000 7fac7affd640 1 -- 172.21.15.3:0/3359862961 learned_addr learned my addr 172.21.15.3:0/3359862961 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T03:47:00.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.918+0000 7fac7a7fc640 1 --2- 172.21.15.3:0/3359862961 >> v2:172.21.15.3:3300/0 conn(0x7fac7c09d0f0 0x7fac7c138ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.918+0000 7fac7a7fc640 1 -- 172.21.15.3:0/3359862961 >> v1:172.21.15.3:6789/0 conn(0x7fac7c09e6c0 legacy=0x7fac7c09c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.918+0000 7fac7a7fc640 1 -- 172.21.15.3:0/3359862961 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7c0a5ee0 con 0x7fac7c09d0f0 2024-09-22T03:47:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.919+0000 7fac7a7fc640 1 --2- 172.21.15.3:0/3359862961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac7c09d0f0 0x7fac7c138ee0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36aab9cd2aa6c25 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.919+0000 7fac79ffb640 1 -- 172.21.15.3:0/3359862961 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fac70009d80 con 0x7fac7c09d0f0 2024-09-22T03:47:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.919+0000 7fac79ffb640 1 -- 172.21.15.3:0/3359862961 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fac7000e230 con 0x7fac7c09d0f0 2024-09-22T03:47:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.919+0000 7fac79ffb640 1 -- 172.21.15.3:0/3359862961 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fac7000e4d0 con 0x7fac7c09d0f0 2024-09-22T03:47:00.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.919+0000 7fac81551640 1 -- 172.21.15.3:0/3359862961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac7c09d0f0 msgr2=0x7fac7c138ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:00.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.919+0000 7fac81551640 1 --2- 172.21.15.3:0/3359862961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac7c09d0f0 0x7fac7c138ee0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.920+0000 7fac81551640 1 -- 172.21.15.3:0/3359862961 shutdown_connections 2024-09-22T03:47:00.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.920+0000 7fac81551640 1 --2- 172.21.15.3:0/3359862961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac7c09d0f0 0x7fac7c138ee0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:00.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.920+0000 7fac81551640 1 -- 172.21.15.3:0/3359862961 >> 172.21.15.3:0/3359862961 conn(0x7fac7c0a43e0 msgr2=0x7fac7c073ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:47:00.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.920+0000 7fac81551640 1 -- 172.21.15.3:0/3359862961 shutdown_connections 2024-09-22T03:47:00.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.920+0000 7fac81551640 1 -- 172.21.15.3:0/3359862961 wait complete. 2024-09-22T03:47:00.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.920+0000 7fac81551640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:47:00.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.922+0000 7fac81551640 1 Processor -- start 2024-09-22T03:47:00.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.922+0000 7fac81551640 1 -- start start 2024-09-22T03:47:00.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.923+0000 7fac81551640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac7c09d0f0 0x7fac7c123c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:00.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.923+0000 7fac81551640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c124630 con 0x7fac7c09d0f0 2024-09-22T03:47:00.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.923+0000 7fac7affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac7c09d0f0 0x7fac7c123c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:00.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.923+0000 7fac7affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac7c09d0f0 0x7fac7c123c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59072/0 (socket says 172.21.15.3:59072) 2024-09-22T03:47:00.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.923+0000 7fac7affd640 1 -- 172.21.15.3:0/3958850732 learned_addr learned my addr 172.21.15.3:0/3958850732 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:47:00.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.924+0000 7fac7affd640 1 -- 172.21.15.3:0/3958850732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7c122550 con 0x7fac7c09d0f0 2024-09-22T03:47:00.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.924+0000 7fac7affd640 1 --2- 172.21.15.3:0/3958850732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac7c09d0f0 0x7fac7c123c00 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:00.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.924+0000 7fac637fe640 1 -- 172.21.15.3:0/3958850732 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fac6c0070b0 con 0x7fac7c09d0f0 2024-09-22T03:47:00.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.924+0000 7fac637fe640 1 -- 172.21.15.3:0/3958850732 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fac6c007210 con 0x7fac7c09d0f0 2024-09-22T03:47:00.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.924+0000 7fac81551640 1 -- 172.21.15.3:0/3958850732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7c124340 con 0x7fac7c09d0f0 2024-09-22T03:47:00.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.924+0000 7fac637fe640 1 -- 172.21.15.3:0/3958850732 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fac6c0074b0 con 0x7fac7c09d0f0 2024-09-22T03:47:00.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.924+0000 7fac81551640 1 -- 172.21.15.3:0/3958850732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac7c1249b0 con 0x7fac7c09d0f0 2024-09-22T03:47:00.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.925+0000 7fac81551640 1 -- 172.21.15.3:0/3958850732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac7c09c9e0 con 0x7fac7c09d0f0 2024-09-22T03:47:00.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.925+0000 7fac637fe640 1 -- 172.21.15.3:0/3958850732 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fac6c007660 con 0x7fac7c09d0f0 2024-09-22T03:47:00.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.925+0000 7fac637fe640 1 --2- 172.21.15.3:0/3958850732 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fac6803ead0 0x7fac68040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:00.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.925+0000 7fac637fe640 1 -- 172.21.15.3:0/3958850732 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7fac6c0501f0 con 0x7fac7c09d0f0 2024-09-22T03:47:00.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.926+0000 7fac7a7fc640 1 --2- 172.21.15.3:0/3958850732 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fac6803ead0 0x7fac68040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:00.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.926+0000 7fac7a7fc640 1 --2- 172.21.15.3:0/3958850732 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fac6803ead0 0x7fac68040fe0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:00.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:00.931+0000 7fac637fe640 1 -- 172.21.15.3:0/3958850732 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7fac6c0096b0 con 0x7fac7c09d0f0 2024-09-22T03:47:01.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.113+0000 7fac81551640 1 -- 172.21.15.3:0/3958850732 --> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac7c09d500 con 0x7fac6803ead0 2024-09-22T03:47:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.115+0000 7fac637fe640 1 -- 172.21.15.3:0/3958850732 <== mgr.4099 v2:172.21.15.123:6800/551784226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+31202 (crc 0 0 0) 0x7fac7000e3a0 con 0x7fac6803ead0 2024-09-22T03:47:01.116 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:47:01.117 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-22T03:47:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.117+0000 7fac81551640 1 -- 172.21.15.3:0/3958850732 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fac6803ead0 msgr2=0x7fac68040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.117+0000 7fac81551640 1 --2- 172.21.15.3:0/3958850732 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fac6803ead0 0x7fac68040fe0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.117+0000 7fac81551640 1 -- 172.21.15.3:0/3958850732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac7c09d0f0 msgr2=0x7fac7c123c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.117+0000 7fac81551640 1 --2- 172.21.15.3:0/3958850732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac7c09d0f0 0x7fac7c123c00 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.117+0000 7fac81551640 1 -- 172.21.15.3:0/3958850732 shutdown_connections 2024-09-22T03:47:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.117+0000 7fac81551640 1 --2- 172.21.15.3:0/3958850732 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fac6803ead0 0x7fac68040fe0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.117+0000 7fac81551640 1 --2- 172.21.15.3:0/3958850732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac7c09d0f0 0x7fac7c123c00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.117+0000 7fac81551640 1 -- 172.21.15.3:0/3958850732 >> 172.21.15.3:0/3958850732 conn(0x7fac7c0a43e0 msgr2=0x7fac7c1bbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:47:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.117+0000 7fac81551640 1 -- 172.21.15.3:0/3958850732 shutdown_connections 2024-09-22T03:47:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.117+0000 7fac81551640 1 -- 172.21.15.3:0/3958850732 wait complete. 2024-09-22T03:47:01.129 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-09-22T03:46:59.557316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":3664,"kb_used_data":3664,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209187248,"statfs":{"total":214211493888,"available":214207741952,"internally_reserved":0,"allocated":3751936,"data_stored":3751936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.195270"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.331054+0000","last_change":"2024-09-22T03:46:57.335250+0000","last_active":"2024-09-22T03:46:57.331054+0000","last_peered":"2024-09-22T03:46:57.331054+0000","last_clean":"2024-09-22T03:46:57.331054+0000","last_became_active":"2024-09-22T03:46:55.378262+0000","last_became_peered":"2024-09-22T03:46:55.378262+0000","last_unstale":"2024-09-22T03:46:57.331054+0000","last_undegraded":"2024-09-22T03:46:57.331054+0000","last_fullsized":"2024-09-22T03:46:57.331054+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.330015+0000","last_change":"2024-09-22T03:46:57.333626+0000","last_active":"2024-09-22T03:46:57.330015+0000","last_peered":"2024-09-22T03:46:57.330015+0000","last_clean":"2024-09-22T03:46:57.330015+0000","last_became_active":"2024-09-22T03:46:55.378604+0000","last_became_peered":"2024-09-22T03:46:55.378604+0000","last_unstale":"2024-09-22T03:46:57.330015+0000","last_undegraded":"2024-09-22T03:46:57.330015+0000","last_fullsized":"2024-09-22T03:46:57.330015+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.325308+0000","last_change":"2024-09-22T03:46:57.329704+0000","last_active":"2024-09-22T03:46:57.325308+0000","last_peered":"2024-09-22T03:46:57.325308+0000","last_clean":"2024-09-22T03:46:57.325308+0000","last_became_active":"2024-09-22T03:46:55.370620+0000","last_became_peered":"2024-09-22T03:46:55.370620+0000","last_unstale":"2024-09-22T03:46:57.325308+0000","last_undegraded":"2024-09-22T03:46:57.325308+0000","last_fullsized":"2024-09-22T03:46:57.325308+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.323051+0000","last_change":"2024-09-22T03:46:57.328548+0000","last_active":"2024-09-22T03:46:57.323051+0000","last_peered":"2024-09-22T03:46:57.323051+0000","last_clean":"2024-09-22T03:46:57.323051+0000","last_became_active":"2024-09-22T03:46:55.358980+0000","last_became_peered":"2024-09-22T03:46:55.358980+0000","last_unstale":"2024-09-22T03:46:57.323051+0000","last_undegraded":"2024-09-22T03:46:57.323051+0000","last_fullsized":"2024-09-22T03:46:57.323051+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.331270+0000","last_change":"2024-09-22T03:46:57.331270+0000","last_active":"2024-09-22T03:46:57.331270+0000","last_peered":"2024-09-22T03:46:57.331270+0000","last_clean":"2024-09-22T03:46:57.331270+0000","last_became_active":"2024-09-22T03:46:55.383021+0000","last_became_peered":"2024-09-22T03:46:55.383021+0000","last_unstale":"2024-09-22T03:46:57.331270+0000","last_undegraded":"2024-09-22T03:46:57.331270+0000","last_fullsized":"2024-09-22T03:46:57.331270+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.325104+0000","last_change":"2024-09-22T03:46:57.329346+0000","last_active":"2024-09-22T03:46:57.325104+0000","last_peered":"2024-09-22T03:46:57.325104+0000","last_clean":"2024-09-22T03:46:57.325104+0000","last_became_active":"2024-09-22T03:46:55.358542+0000","last_became_peered":"2024-09-22T03:46:55.358542+0000","last_unstale":"2024-09-22T03:46:57.325104+0000","last_undegraded":"2024-09-22T03:46:57.325104+0000","last_fullsized":"2024-09-22T03:46:57.325104+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.327293+0000","last_change":"2024-09-22T03:46:57.332056+0000","last_active":"2024-09-22T03:46:57.327293+0000","last_peered":"2024-09-22T03:46:57.327293+0000","last_clean":"2024-09-22T03:46:57.327293+0000","last_became_active":"2024-09-22T03:46:55.412604+0000","last_became_peered":"2024-09-22T03:46:55.412604+0000","last_unstale":"2024-09-22T03:46:57.327293+0000","last_undegraded":"2024-09-22T03:46:57.327293+0000","last_fullsized":"2024-09-22T03:46:57.327293+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.330641+0000","last_change":"2024-09-22T03:46:57.335158+0000","last_active":"2024-09-22T03:46:57.330641+0000","last_peered":"2024-09-22T03:46:57.330641+0000","last_clean":"2024-09-22T03:46:57.330641+0000","last_became_active":"2024-09-22T03:46:55.387128+0000","last_became_peered":"2024-09-22T03:46:55.387128+0000","last_unstale":"2024-09-22T03:46:57.330641+0000","last_undegraded":"2024-09-22T03:46:57.330641+0000","last_fullsized":"2024-09-22T03:46:57.330641+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":87,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.322525+0000","last_change":"2024-09-22T03:46:53.370744+0000","last_active":"2024-09-22T03:46:57.322525+0000","last_peered":"2024-09-22T03:46:57.322525+0000","last_clean":"2024-09-22T03:46:57.322525+0000","last_became_active":"2024-09-22T03:46:53.349788+0000","last_became_peered":"2024-09-22T03:46:53.349788+0000","last_unstale":"2024-09-22T03:46:57.322525+0000","last_undegraded":"2024-09-22T03:46:57.322525+0000","last_fullsized":"2024-09-22T03:46:57.322525+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:52.275666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:52.275666+0000","last_clean_scrub_stamp":"2024-09-22T03:46:52.275666+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214207741952,"internally_reserved":0,"allocated":3751936,"data_stored":3751936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53551673344,"internally_reserved":0,"allocated":1200128,"data_stored":1200128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705672,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1172,"kb_used_data":1172,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296556,"statfs":{"total":53552873472,"available":53551673344,"internally_reserved":0,"allocated":1200128,"data_stored":1200128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705672,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":532,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52297196,"statfs":{"total":53552873472,"available":53552328704,"internally_reserved":0,"allocated":544768,"data_stored":544768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705672,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":528,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52297200,"statfs":{"total":53552873472,"available":53552332800,"internally_reserved":0,"allocated":540672,"data_stored":540672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705672,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1432,"kb_used_data":1432,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296296,"statfs":{"total":53552873472,"available":53551407104,"internally_reserved":0,"allocated":1466368,"data_stored":1466368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53551673344,"internally_reserved":0,"allocated":1200128,"data_stored":1200128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53551407104,"internally_reserved":0,"allocated":1466368,"data_stored":1466368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53552332800,"internally_reserved":0,"allocated":540672,"data_stored":540672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53552328704,"internally_reserved":0,"allocated":544768,"data_stored":544768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53551673344,"internally_reserved":0,"allocated":1200128,"data_stored":1200128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:47:01.130 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T03:47:01.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.237+0000 7f616f791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:47:01.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.242+0000 7f616f791640 1 Processor -- start 2024-09-22T03:47:01.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.242+0000 7f616f791640 1 -- start start 2024-09-22T03:47:01.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.244+0000 7f616f791640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f616809d0f0 0x7f6168138ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:01.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.244+0000 7f616f791640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6168074a70 con 0x7f616809e6c0 2024-09-22T03:47:01.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.244+0000 7f616f791640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f61680a61f0 con 0x7f616809d0f0 2024-09-22T03:47:01.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.244+0000 7f616cd2c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f616809d0f0 0x7f6168138ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:01.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.244+0000 7f616cd2c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f616809d0f0 0x7f6168138ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59076/0 (socket says 172.21.15.3:59076) 2024-09-22T03:47:01.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.244+0000 7f616d52d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f616809e6c0 0x7f616809c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33268/0 (socket says 172.21.15.3:33268) 2024-09-22T03:47:01.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.244+0000 7f616cd2c640 1 -- 172.21.15.3:0/3852446489 learned_addr learned my addr 172.21.15.3:0/3852446489 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:47:01.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.244+0000 7f616cd2c640 1 -- 172.21.15.3:0/3852446489 >> v1:172.21.15.3:6789/0 conn(0x7f616809e6c0 legacy=0x7f616809c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:01.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.244+0000 7f616cd2c640 1 -- 172.21.15.3:0/3852446489 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61680a5ee0 con 0x7f616809d0f0 2024-09-22T03:47:01.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.244+0000 7f6167fff640 1 -- 172.21.15.3:0/3852446489 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1763603181 0 0) 0x7f6168074a70 con 0x7f616809e6c0 2024-09-22T03:47:01.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.245+0000 7f616cd2c640 1 --2- 172.21.15.3:0/3852446489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f616809d0f0 0x7f6168138ee0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f714e0771c5474 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:01.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.245+0000 7f6167fff640 1 -- 172.21.15.3:0/3852446489 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f615c009e40 con 0x7f616809d0f0 2024-09-22T03:47:01.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.245+0000 7f6167fff640 1 -- 172.21.15.3:0/3852446489 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f615c00e230 con 0x7f616809d0f0 2024-09-22T03:47:01.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.245+0000 7f6167fff640 1 -- 172.21.15.3:0/3852446489 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f615c00e4d0 con 0x7f616809d0f0 2024-09-22T03:47:01.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.245+0000 7f616f791640 1 -- 172.21.15.3:0/3852446489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f616809d0f0 msgr2=0x7f6168138ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:01.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.245+0000 7f616f791640 1 --2- 172.21.15.3:0/3852446489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f616809d0f0 0x7f6168138ee0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:01.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.246+0000 7f616f791640 1 -- 172.21.15.3:0/3852446489 shutdown_connections 2024-09-22T03:47:01.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.246+0000 7f616f791640 1 --2- 172.21.15.3:0/3852446489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f616809d0f0 0x7f6168138ee0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:01.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.246+0000 7f616f791640 1 -- 172.21.15.3:0/3852446489 >> 172.21.15.3:0/3852446489 conn(0x7f61680a43e0 msgr2=0x7f6168073ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:47:01.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.246+0000 7f616f791640 1 -- 172.21.15.3:0/3852446489 shutdown_connections 2024-09-22T03:47:01.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.246+0000 7f616f791640 1 -- 172.21.15.3:0/3852446489 wait complete. 2024-09-22T03:47:01.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.246+0000 7f616f791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:47:01.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.248+0000 7f616f791640 1 Processor -- start 2024-09-22T03:47:01.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.248+0000 7f616f791640 1 -- start start 2024-09-22T03:47:01.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.249+0000 7f616f791640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f616809d0f0 0x7f61681c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:01.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.249+0000 7f616f791640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6168125810 con 0x7f616809d0f0 2024-09-22T03:47:01.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.249+0000 7f616d52d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f616809d0f0 0x7f61681c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:01.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.249+0000 7f616d52d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f616809d0f0 0x7f61681c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59086/0 (socket says 172.21.15.3:59086) 2024-09-22T03:47:01.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.249+0000 7f616d52d640 1 -- 172.21.15.3:0/485094435 learned_addr learned my addr 172.21.15.3:0/485094435 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:47:01.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.250+0000 7f616d52d640 1 -- 172.21.15.3:0/485094435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61681218a0 con 0x7f616809d0f0 2024-09-22T03:47:01.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.250+0000 7f616d52d640 1 --2- 172.21.15.3:0/485094435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f616809d0f0 0x7f61681c8390 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:01.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.250+0000 7f6165ffb640 1 -- 172.21.15.3:0/485094435 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f61580070b0 con 0x7f616809d0f0 2024-09-22T03:47:01.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.250+0000 7f6165ffb640 1 -- 172.21.15.3:0/485094435 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f6158007210 con 0x7f616809d0f0 2024-09-22T03:47:01.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.250+0000 7f616f791640 1 -- 172.21.15.3:0/485094435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61681242d0 con 0x7f616809d0f0 2024-09-22T03:47:01.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.250+0000 7f6165ffb640 1 -- 172.21.15.3:0/485094435 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f61580074b0 con 0x7f616809d0f0 2024-09-22T03:47:01.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.250+0000 7f616f791640 1 -- 172.21.15.3:0/485094435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6168121b40 con 0x7f616809d0f0 2024-09-22T03:47:01.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.251+0000 7f6165ffb640 1 -- 172.21.15.3:0/485094435 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f6158007660 con 0x7f616809d0f0 2024-09-22T03:47:01.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.251+0000 7f616f791640 1 -- 172.21.15.3:0/485094435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f616809c9e0 con 0x7f616809d0f0 2024-09-22T03:47:01.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.251+0000 7f6165ffb640 1 --2- 172.21.15.3:0/485094435 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f615403ead0 0x7f6154040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:01.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.251+0000 7f6165ffb640 1 -- 172.21.15.3:0/485094435 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7f61580503d0 con 0x7f616809d0f0 2024-09-22T03:47:01.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.252+0000 7f616cd2c640 1 --2- 172.21.15.3:0/485094435 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f615403ead0 0x7f6154040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:01.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.252+0000 7f616cd2c640 1 --2- 172.21.15.3:0/485094435 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f615403ead0 0x7f6154040fe0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:01.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.257+0000 7f6165ffb640 1 -- 172.21.15.3:0/485094435 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f6158009830 con 0x7f616809d0f0 2024-09-22T03:47:01.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.452+0000 7f616f791640 1 -- 172.21.15.3:0/485094435 --> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f616809d500 con 0x7f615403ead0 2024-09-22T03:47:01.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.454+0000 7f6165ffb640 1 -- 172.21.15.3:0/485094435 <== mgr.4099 v2:172.21.15.123:6800/551784226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+31202 (crc 0 0 0) 0x7f615c009e10 con 0x7f615403ead0 2024-09-22T03:47:01.455 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:47:01.455 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-22T03:47:01.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.455+0000 7f616f791640 1 -- 172.21.15.3:0/485094435 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f615403ead0 msgr2=0x7f6154040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:01.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.456+0000 7f616f791640 1 --2- 172.21.15.3:0/485094435 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f615403ead0 0x7f6154040fe0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:01.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.456+0000 7f616f791640 1 -- 172.21.15.3:0/485094435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f616809d0f0 msgr2=0x7f61681c8390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:01.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.456+0000 7f616f791640 1 --2- 172.21.15.3:0/485094435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f616809d0f0 0x7f61681c8390 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:01.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.456+0000 7f616f791640 1 -- 172.21.15.3:0/485094435 shutdown_connections 2024-09-22T03:47:01.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.456+0000 7f616f791640 1 --2- 172.21.15.3:0/485094435 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f615403ead0 0x7f6154040fe0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:01.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.456+0000 7f616f791640 1 --2- 172.21.15.3:0/485094435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f616809d0f0 0x7f61681c8390 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:01.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.456+0000 7f616f791640 1 -- 172.21.15.3:0/485094435 >> 172.21.15.3:0/485094435 conn(0x7f61680a43e0 msgr2=0x7f61680a0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:47:01.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.456+0000 7f616f791640 1 -- 172.21.15.3:0/485094435 shutdown_connections 2024-09-22T03:47:01.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.456+0000 7f616f791640 1 -- 172.21.15.3:0/485094435 wait complete. 2024-09-22T03:47:01.468 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-09-22T03:46:59.557316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":3664,"kb_used_data":3664,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209187248,"statfs":{"total":214211493888,"available":214207741952,"internally_reserved":0,"allocated":3751936,"data_stored":3751936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.195270"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.331054+0000","last_change":"2024-09-22T03:46:57.335250+0000","last_active":"2024-09-22T03:46:57.331054+0000","last_peered":"2024-09-22T03:46:57.331054+0000","last_clean":"2024-09-22T03:46:57.331054+0000","last_became_active":"2024-09-22T03:46:55.378262+0000","last_became_peered":"2024-09-22T03:46:55.378262+0000","last_unstale":"2024-09-22T03:46:57.331054+0000","last_undegraded":"2024-09-22T03:46:57.331054+0000","last_fullsized":"2024-09-22T03:46:57.331054+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.330015+0000","last_change":"2024-09-22T03:46:57.333626+0000","last_active":"2024-09-22T03:46:57.330015+0000","last_peered":"2024-09-22T03:46:57.330015+0000","last_clean":"2024-09-22T03:46:57.330015+0000","last_became_active":"2024-09-22T03:46:55.378604+0000","last_became_peered":"2024-09-22T03:46:55.378604+0000","last_unstale":"2024-09-22T03:46:57.330015+0000","last_undegraded":"2024-09-22T03:46:57.330015+0000","last_fullsized":"2024-09-22T03:46:57.330015+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.325308+0000","last_change":"2024-09-22T03:46:57.329704+0000","last_active":"2024-09-22T03:46:57.325308+0000","last_peered":"2024-09-22T03:46:57.325308+0000","last_clean":"2024-09-22T03:46:57.325308+0000","last_became_active":"2024-09-22T03:46:55.370620+0000","last_became_peered":"2024-09-22T03:46:55.370620+0000","last_unstale":"2024-09-22T03:46:57.325308+0000","last_undegraded":"2024-09-22T03:46:57.325308+0000","last_fullsized":"2024-09-22T03:46:57.325308+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.323051+0000","last_change":"2024-09-22T03:46:57.328548+0000","last_active":"2024-09-22T03:46:57.323051+0000","last_peered":"2024-09-22T03:46:57.323051+0000","last_clean":"2024-09-22T03:46:57.323051+0000","last_became_active":"2024-09-22T03:46:55.358980+0000","last_became_peered":"2024-09-22T03:46:55.358980+0000","last_unstale":"2024-09-22T03:46:57.323051+0000","last_undegraded":"2024-09-22T03:46:57.323051+0000","last_fullsized":"2024-09-22T03:46:57.323051+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.331270+0000","last_change":"2024-09-22T03:46:57.331270+0000","last_active":"2024-09-22T03:46:57.331270+0000","last_peered":"2024-09-22T03:46:57.331270+0000","last_clean":"2024-09-22T03:46:57.331270+0000","last_became_active":"2024-09-22T03:46:55.383021+0000","last_became_peered":"2024-09-22T03:46:55.383021+0000","last_unstale":"2024-09-22T03:46:57.331270+0000","last_undegraded":"2024-09-22T03:46:57.331270+0000","last_fullsized":"2024-09-22T03:46:57.331270+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.325104+0000","last_change":"2024-09-22T03:46:57.329346+0000","last_active":"2024-09-22T03:46:57.325104+0000","last_peered":"2024-09-22T03:46:57.325104+0000","last_clean":"2024-09-22T03:46:57.325104+0000","last_became_active":"2024-09-22T03:46:55.358542+0000","last_became_peered":"2024-09-22T03:46:55.358542+0000","last_unstale":"2024-09-22T03:46:57.325104+0000","last_undegraded":"2024-09-22T03:46:57.325104+0000","last_fullsized":"2024-09-22T03:46:57.325104+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.327293+0000","last_change":"2024-09-22T03:46:57.332056+0000","last_active":"2024-09-22T03:46:57.327293+0000","last_peered":"2024-09-22T03:46:57.327293+0000","last_clean":"2024-09-22T03:46:57.327293+0000","last_became_active":"2024-09-22T03:46:55.412604+0000","last_became_peered":"2024-09-22T03:46:55.412604+0000","last_unstale":"2024-09-22T03:46:57.327293+0000","last_undegraded":"2024-09-22T03:46:57.327293+0000","last_fullsized":"2024-09-22T03:46:57.327293+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.330641+0000","last_change":"2024-09-22T03:46:57.335158+0000","last_active":"2024-09-22T03:46:57.330641+0000","last_peered":"2024-09-22T03:46:57.330641+0000","last_clean":"2024-09-22T03:46:57.330641+0000","last_became_active":"2024-09-22T03:46:55.387128+0000","last_became_peered":"2024-09-22T03:46:55.387128+0000","last_unstale":"2024-09-22T03:46:57.330641+0000","last_undegraded":"2024-09-22T03:46:57.330641+0000","last_fullsized":"2024-09-22T03:46:57.330641+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":87,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-22T03:46:57.322525+0000","last_change":"2024-09-22T03:46:53.370744+0000","last_active":"2024-09-22T03:46:57.322525+0000","last_peered":"2024-09-22T03:46:57.322525+0000","last_clean":"2024-09-22T03:46:57.322525+0000","last_became_active":"2024-09-22T03:46:53.349788+0000","last_became_peered":"2024-09-22T03:46:53.349788+0000","last_unstale":"2024-09-22T03:46:57.322525+0000","last_undegraded":"2024-09-22T03:46:57.322525+0000","last_fullsized":"2024-09-22T03:46:57.322525+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:52.275666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:52.275666+0000","last_clean_scrub_stamp":"2024-09-22T03:46:52.275666+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214207741952,"internally_reserved":0,"allocated":3751936,"data_stored":3751936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53551673344,"internally_reserved":0,"allocated":1200128,"data_stored":1200128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705672,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1172,"kb_used_data":1172,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296556,"statfs":{"total":53552873472,"available":53551673344,"internally_reserved":0,"allocated":1200128,"data_stored":1200128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705672,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":532,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52297196,"statfs":{"total":53552873472,"available":53552328704,"internally_reserved":0,"allocated":544768,"data_stored":544768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705672,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":528,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52297200,"statfs":{"total":53552873472,"available":53552332800,"internally_reserved":0,"allocated":540672,"data_stored":540672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705672,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1432,"kb_used_data":1432,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296296,"statfs":{"total":53552873472,"available":53551407104,"internally_reserved":0,"allocated":1466368,"data_stored":1466368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53551673344,"internally_reserved":0,"allocated":1200128,"data_stored":1200128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53551407104,"internally_reserved":0,"allocated":1466368,"data_stored":1466368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53552332800,"internally_reserved":0,"allocated":540672,"data_stored":540672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53552328704,"internally_reserved":0,"allocated":544768,"data_stored":544768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53551673344,"internally_reserved":0,"allocated":1200128,"data_stored":1200128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T03:47:01.468 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-22T03:47:01.468 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-22T03:47:01.469 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-22T03:47:01.469 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-22T03:47:01.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.575+0000 7fd940351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:47:01.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.579+0000 7fd940351640 1 Processor -- start 2024-09-22T03:47:01.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.579+0000 7fd940351640 1 -- start start 2024-09-22T03:47:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.581+0000 7fd940351640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd938070d30 0x7fd9380731f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.581+0000 7fd940351640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd93806fa20 con 0x7fd938073730 2024-09-22T03:47:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.581+0000 7fd940351640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd938071710 con 0x7fd938070d30 2024-09-22T03:47:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.581+0000 7fd93e0ed640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd938070d30 0x7fd9380731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.581+0000 7fd93e0ed640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd938070d30 0x7fd9380731f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59094/0 (socket says 172.21.15.3:59094) 2024-09-22T03:47:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.581+0000 7fd93d8ec640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd938073730 0x7fd938135da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33278/0 (socket says 172.21.15.3:33278) 2024-09-22T03:47:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.581+0000 7fd93e0ed640 1 -- 172.21.15.3:0/2797995981 learned_addr learned my addr 172.21.15.3:0/2797995981 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:47:01.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.582+0000 7fd93e0ed640 1 -- 172.21.15.3:0/2797995981 >> v1:172.21.15.3:6789/0 conn(0x7fd938073730 legacy=0x7fd938135da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:01.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.582+0000 7fd93d0eb640 1 -- 172.21.15.3:0/2797995981 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2967583206 0 0) 0x7fd93806fa20 con 0x7fd938073730 2024-09-22T03:47:01.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.582+0000 7fd93e0ed640 1 -- 172.21.15.3:0/2797995981 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93806ff50 con 0x7fd938070d30 2024-09-22T03:47:01.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.582+0000 7fd93e0ed640 1 --2- 172.21.15.3:0/2797995981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd938070d30 0x7fd9380731f0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd0f9660209a88c server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:01.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.582+0000 7fd93d0eb640 1 -- 172.21.15.3:0/2797995981 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd934009e40 con 0x7fd938070d30 2024-09-22T03:47:01.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.582+0000 7fd93d0eb640 1 -- 172.21.15.3:0/2797995981 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fd93400e230 con 0x7fd938070d30 2024-09-22T03:47:01.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.582+0000 7fd93d0eb640 1 -- 172.21.15.3:0/2797995981 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd93400e4d0 con 0x7fd938070d30 2024-09-22T03:47:01.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.583+0000 7fd940351640 1 -- 172.21.15.3:0/2797995981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd938070d30 msgr2=0x7fd9380731f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:01.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.583+0000 7fd940351640 1 --2- 172.21.15.3:0/2797995981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd938070d30 0x7fd9380731f0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:01.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.583+0000 7fd940351640 1 -- 172.21.15.3:0/2797995981 shutdown_connections 2024-09-22T03:47:01.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.583+0000 7fd940351640 1 --2- 172.21.15.3:0/2797995981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd938070d30 0x7fd9380731f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:01.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.583+0000 7fd940351640 1 -- 172.21.15.3:0/2797995981 >> 172.21.15.3:0/2797995981 conn(0x7fd9380a6d10 msgr2=0x7fd938071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:47:01.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.583+0000 7fd940351640 1 -- 172.21.15.3:0/2797995981 shutdown_connections 2024-09-22T03:47:01.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.583+0000 7fd940351640 1 -- 172.21.15.3:0/2797995981 wait complete. 2024-09-22T03:47:01.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.583+0000 7fd940351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:47:01.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.585+0000 7fd940351640 1 Processor -- start 2024-09-22T03:47:01.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.585+0000 7fd940351640 1 -- start start 2024-09-22T03:47:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.586+0000 7fd940351640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd938070d30 0x7fd9381240f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.586+0000 7fd940351640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd938123ca0 con 0x7fd938070d30 2024-09-22T03:47:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.586+0000 7fd93e0ed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd938070d30 0x7fd9381240f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.586+0000 7fd93e0ed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd938070d30 0x7fd9381240f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59110/0 (socket says 172.21.15.3:59110) 2024-09-22T03:47:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.586+0000 7fd93e0ed640 1 -- 172.21.15.3:0/2984644158 learned_addr learned my addr 172.21.15.3:0/2984644158 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:47:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.587+0000 7fd93e0ed640 1 -- 172.21.15.3:0/2984644158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd938125120 con 0x7fd938070d30 2024-09-22T03:47:01.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.587+0000 7fd93e0ed640 1 --2- 172.21.15.3:0/2984644158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd938070d30 0x7fd9381240f0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.587+0000 7fd92effd640 1 -- 172.21.15.3:0/2984644158 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd93400ee10 con 0x7fd938070d30 2024-09-22T03:47:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.587+0000 7fd92effd640 1 -- 172.21.15.3:0/2984644158 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fd934015420 con 0x7fd938070d30 2024-09-22T03:47:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.587+0000 7fd940351640 1 -- 172.21.15.3:0/2984644158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9381236a0 con 0x7fd938070d30 2024-09-22T03:47:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.587+0000 7fd92effd640 1 -- 172.21.15.3:0/2984644158 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fd9340156c0 con 0x7fd938070d30 2024-09-22T03:47:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.587+0000 7fd940351640 1 -- 172.21.15.3:0/2984644158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd938125ff0 con 0x7fd938070d30 2024-09-22T03:47:01.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.588+0000 7fd940351640 1 -- 172.21.15.3:0/2984644158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9380731f0 con 0x7fd938070d30 2024-09-22T03:47:01.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.588+0000 7fd92effd640 1 -- 172.21.15.3:0/2984644158 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fd9340158c0 con 0x7fd938070d30 2024-09-22T03:47:01.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.588+0000 7fd92effd640 1 --2- 172.21.15.3:0/2984644158 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fd91c03ea80 0x7fd91c040f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:47:01.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.588+0000 7fd92effd640 1 -- 172.21.15.3:0/2984644158 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2728+0+0 (crc 0 0 0) 0x7fd934050fc0 con 0x7fd938070d30 2024-09-22T03:47:01.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.589+0000 7fd93d8ec640 1 --2- 172.21.15.3:0/2984644158 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fd91c03ea80 0x7fd91c040f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:47:01.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.589+0000 7fd93d8ec640 1 --2- 172.21.15.3:0/2984644158 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fd91c03ea80 0x7fd91c040f90 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:47:01.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.593+0000 7fd92effd640 1 -- 172.21.15.3:0/2984644158 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7fd934010a00 con 0x7fd938070d30 2024-09-22T03:47:01.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.818+0000 7fd940351640 1 -- 172.21.15.3:0/2984644158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd938077360 con 0x7fd938070d30 2024-09-22T03:47:01.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.819+0000 7fd92effd640 1 -- 172.21.15.3:0/2984644158 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (crc 0 0 0) 0x7fd934010310 con 0x7fd938070d30 2024-09-22T03:47:01.820 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T03:47:01.820 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-22T03:47:01.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.821+0000 7fd940351640 1 -- 172.21.15.3:0/2984644158 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fd91c03ea80 msgr2=0x7fd91c040f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:01.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.821+0000 7fd940351640 1 --2- 172.21.15.3:0/2984644158 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fd91c03ea80 0x7fd91c040f90 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:01.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.821+0000 7fd940351640 1 -- 172.21.15.3:0/2984644158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd938070d30 msgr2=0x7fd9381240f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:47:01.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.821+0000 7fd940351640 1 --2- 172.21.15.3:0/2984644158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd938070d30 0x7fd9381240f0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:01.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.821+0000 7fd940351640 1 -- 172.21.15.3:0/2984644158 shutdown_connections 2024-09-22T03:47:01.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.821+0000 7fd940351640 1 --2- 172.21.15.3:0/2984644158 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fd91c03ea80 0x7fd91c040f90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:01.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.821+0000 7fd940351640 1 --2- 172.21.15.3:0/2984644158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd938070d30 0x7fd9381240f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:47:01.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.821+0000 7fd940351640 1 -- 172.21.15.3:0/2984644158 >> 172.21.15.3:0/2984644158 conn(0x7fd9380a6d10 msgr2=0x7fd93811cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:47:01.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.821+0000 7fd940351640 1 -- 172.21.15.3:0/2984644158 shutdown_connections 2024-09-22T03:47:01.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T03:47:01.821+0000 7fd940351640 1 -- 172.21.15.3:0/2984644158 wait complete. 2024-09-22T03:47:01.834 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-22T03:47:01.834 INFO:teuthology.run_tasks:Running task workunit... 2024-09-22T03:47:01.845 INFO:tasks.workunit:Pulling workunits from ref 62da54b4b83a96ca9976b7e1d0c8272a564a2208 2024-09-22T03:47:01.845 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-22T03:47:01.846 DEBUG:teuthology.orchestra.run.smithi003:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-22T03:47:01.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T03:47:01.861 INFO:teuthology.orchestra.run.smithi003.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-22T03:47:01.862 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-22T03:47:01.917 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-22T03:47:01.917 DEBUG:teuthology.orchestra.run.smithi003:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-22T03:47:01.973 INFO:tasks.workunit:timeout=3h 2024-09-22T03:47:01.973 INFO:tasks.workunit:cleanup=True 2024-09-22T03:47:01.973 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 62da54b4b83a96ca9976b7e1d0c8272a564a2208 2024-09-22T03:47:02.029 INFO:tasks.workunit.client.0.smithi003.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-22T03:47:58.492 INFO:tasks.workunit.client.0.smithi003.stderr:Updating files: 93% (12072/12896) Updating files: 94% (12123/12896) Updating files: 95% (12252/12896) Updating files: 96% (12381/12896) Updating files: 97% (12510/12896) Updating files: 98% (12639/12896) Updating files: 99% (12768/12896) Updating files: 100% (12896/12896) Updating files: 100% (12896/12896), done. 2024-09-22T03:47:58.947 INFO:tasks.workunit.client.0.smithi003.stderr:Note: switching to '62da54b4b83a96ca9976b7e1d0c8272a564a2208'. 2024-09-22T03:47:58.947 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-22T03:47:58.947 INFO:tasks.workunit.client.0.smithi003.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-22T03:47:58.947 INFO:tasks.workunit.client.0.smithi003.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-22T03:47:58.947 INFO:tasks.workunit.client.0.smithi003.stderr:state without impacting any branches by switching back to a branch. 2024-09-22T03:47:58.947 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-22T03:47:58.947 INFO:tasks.workunit.client.0.smithi003.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-22T03:47:58.947 INFO:tasks.workunit.client.0.smithi003.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-22T03:47:58.947 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-22T03:47:58.947 INFO:tasks.workunit.client.0.smithi003.stderr: git switch -c 2024-09-22T03:47:58.947 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-22T03:47:58.947 INFO:tasks.workunit.client.0.smithi003.stderr:Or undo this operation with: 2024-09-22T03:47:58.948 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-22T03:47:58.948 INFO:tasks.workunit.client.0.smithi003.stderr: git switch - 2024-09-22T03:47:58.948 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-22T03:47:58.948 INFO:tasks.workunit.client.0.smithi003.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-22T03:47:58.948 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-22T03:47:58.948 INFO:tasks.workunit.client.0.smithi003.stderr:HEAD is now at 62da54b4b83 Merge pull request #59904 from mcv21/patch-2 2024-09-22T03:47:58.949 DEBUG:teuthology.orchestra.run.smithi003:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-22T03:47:59.012 INFO:tasks.workunit.client.0.smithi003.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-22T03:47:59.015 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-22T03:47:59.015 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-22T03:47:59.715 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-22T03:47:59.798 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-22T03:47:59.839 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-22T03:47:59.841 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-22T03:47:59.841 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-22T03:47:59.880 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-22T03:47:59.884 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-22T03:47:59.884 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-22T03:47:59.940 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-09-22T03:47:59.941 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-09-22T03:47:59.941 DEBUG:teuthology.orchestra.run.smithi003:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=62da54b4b83a96ca9976b7e1d0c8272a564a2208 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson 2024-09-22T03:48:00.001 INFO:tasks.workunit.client.0.smithi003.stderr:+ parallel=1 2024-09-22T03:48:00.001 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' --crimson = --serial ']' 2024-09-22T03:48:00.001 INFO:tasks.workunit.client.0.smithi003.stderr:+ crimson=0 2024-09-22T03:48:00.001 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' --crimson = --crimson ']' 2024-09-22T03:48:00.001 INFO:tasks.workunit.client.0.smithi003.stderr:+ parallel=0 2024-09-22T03:48:00.001 INFO:tasks.workunit.client.0.smithi003.stderr:+ crimson=1 2024-09-22T03:48:00.001 INFO:tasks.workunit.client.0.smithi003.stderr:+ color= 2024-09-22T03:48:00.001 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' -t 1 ']' 2024-09-22T03:48:00.002 INFO:tasks.workunit.client.0.smithi003.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-09-22T03:48:00.002 INFO:tasks.workunit.client.0.smithi003.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-09-22T03:48:00.002 INFO:tasks.workunit.client.0.smithi003.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-09-22T03:48:00.002 INFO:tasks.workunit.client.0.smithi003.stderr:+ declare -A pids 2024-09-22T03:48:00.002 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:48:00.002 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:48:00.003 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_aio 2024-09-22T03:48:00.019 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:48:00.019 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 42 tests from 2 test suites. 2024-09-22T03:48:00.019 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:48:00.019 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 26 tests from LibRadosAio 2024-09-22T03:48:00.020 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.TooBig 2024-09-22T03:48:02.206 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.TooBig (2187 ms) 2024-09-22T03:48:02.206 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.SimpleWrite 2024-09-22T03:48:05.212 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.SimpleWrite (3006 ms) 2024-09-22T03:48:05.212 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.WaitForSafe 2024-09-22T03:48:08.226 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.WaitForSafe (3014 ms) 2024-09-22T03:48:08.226 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RoundTrip 2024-09-22T03:48:11.235 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RoundTrip (3009 ms) 2024-09-22T03:48:11.236 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RoundTrip2 2024-09-22T03:48:14.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RoundTrip2 (3027 ms) 2024-09-22T03:48:14.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RoundTrip3 2024-09-22T03:48:17.272 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RoundTrip3 (3010 ms) 2024-09-22T03:48:17.272 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RoundTripAppend 2024-09-22T03:48:20.298 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RoundTripAppend (3026 ms) 2024-09-22T03:48:20.299 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RemoveTest 2024-09-22T03:48:23.312 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RemoveTest (3014 ms) 2024-09-22T03:48:23.313 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.XattrsRoundTrip 2024-09-22T03:48:26.330 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.XattrsRoundTrip (3018 ms) 2024-09-22T03:48:26.331 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RmXattr 2024-09-22T03:48:29.342 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RmXattr (3011 ms) 2024-09-22T03:48:29.342 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.XattrIter 2024-09-22T03:48:32.357 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.XattrIter (3016 ms) 2024-09-22T03:48:32.357 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.IsComplete 2024-09-22T03:48:35.373 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.IsComplete (3016 ms) 2024-09-22T03:48:35.374 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.IsSafe 2024-09-22T03:48:38.386 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.IsSafe (3013 ms) 2024-09-22T03:48:38.386 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.ReturnValue 2024-09-22T03:48:41.401 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.ReturnValue (3015 ms) 2024-09-22T03:48:41.401 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.Flush 2024-09-22T03:48:44.418 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.Flush (3017 ms) 2024-09-22T03:48:44.418 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.FlushAsync 2024-09-22T03:48:47.433 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.FlushAsync (3015 ms) 2024-09-22T03:48:47.434 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RoundTripWriteFull 2024-09-22T03:48:50.449 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RoundTripWriteFull (3016 ms) 2024-09-22T03:48:50.449 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RoundTripWriteSame 2024-09-22T03:48:53.177 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RoundTripWriteSame (2727 ms) 2024-09-22T03:48:53.177 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.SimpleStat 2024-09-22T03:48:56.194 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.SimpleStat (3018 ms) 2024-09-22T03:48:56.195 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.OperateMtime 2024-09-22T03:48:59.206 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.OperateMtime (3012 ms) 2024-09-22T03:48:59.207 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.Operate2Mtime 2024-09-22T03:49:02.225 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.Operate2Mtime (3019 ms) 2024-09-22T03:49:02.226 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.SimpleStatNS 2024-09-22T03:49:05.238 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.SimpleStatNS (3011 ms) 2024-09-22T03:49:05.239 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.StatRemove 2024-09-22T03:49:08.180 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.StatRemove (2944 ms) 2024-09-22T03:49:08.181 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.ExecuteClass 2024-09-22T03:49:11.193 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.ExecuteClass (3013 ms) 2024-09-22T03:49:11.193 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.MultiWrite 2024-09-22T03:49:14.215 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.MultiWrite (3021 ms) 2024-09-22T03:49:14.215 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.AioUnlock 2024-09-22T03:49:17.227 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.AioUnlock (3011 ms) 2024-09-22T03:49:17.227 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 26 tests from LibRadosAio (77206 ms total) 2024-09-22T03:49:17.227 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:49:17.227 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 16 tests from LibRadosAioEC 2024-09-22T03:49:17.227 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.SimpleWrite 2024-09-22T03:49:17.228 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio.cc:1100: Skipped 2024-09-22T03:49:17.228 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:49:17.228 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWrite (0 ms) 2024-09-22T03:49:17.228 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.WaitForComplete 2024-09-22T03:49:17.228 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio.cc:1132: Skipped 2024-09-22T03:49:17.228 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:49:17.228 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.WaitForComplete (0 ms) 2024-09-22T03:49:17.228 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.RoundTrip 2024-09-22T03:49:17.228 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio.cc:1149: Skipped 2024-09-22T03:49:17.228 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:49:17.228 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip (0 ms) 2024-09-22T03:49:17.228 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.RoundTrip2 2024-09-22T03:49:17.229 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio.cc:1182: Skipped 2024-09-22T03:49:17.229 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:49:17.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip2 (0 ms) 2024-09-22T03:49:17.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.RoundTripAppend 2024-09-22T03:49:17.229 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio.cc:1215: Skipped 2024-09-22T03:49:17.229 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:49:17.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppend (0 ms) 2024-09-22T03:49:17.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.IsComplete 2024-09-22T03:49:17.229 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio.cc:1286: Skipped 2024-09-22T03:49:17.229 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:49:17.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.IsComplete (0 ms) 2024-09-22T03:49:17.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.IsSafe 2024-09-22T03:49:17.230 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio.cc:1326: Skipped 2024-09-22T03:49:17.230 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:49:17.230 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.IsSafe (0 ms) 2024-09-22T03:49:17.230 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.ReturnValue 2024-09-22T03:49:17.230 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio.cc:1366: Skipped 2024-09-22T03:49:17.230 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:49:17.230 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValue (0 ms) 2024-09-22T03:49:17.230 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.Flush 2024-09-22T03:49:17.230 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio.cc:1385: Skipped 2024-09-22T03:49:17.230 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:49:17.230 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.Flush (0 ms) 2024-09-22T03:49:17.231 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.FlushAsync 2024-09-22T03:49:17.231 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio.cc:1415: Skipped 2024-09-22T03:49:17.231 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:49:17.231 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsync (0 ms) 2024-09-22T03:49:17.231 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-09-22T03:49:17.231 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio.cc:1454: Skipped 2024-09-22T03:49:17.231 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:49:17.231 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFull (0 ms) 2024-09-22T03:49:17.231 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.SimpleStat 2024-09-22T03:49:17.231 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio.cc:1498: Skipped 2024-09-22T03:49:17.231 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:49:17.232 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStat (0 ms) 2024-09-22T03:49:17.232 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.SimpleStatNS 2024-09-22T03:49:17.232 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio.cc:1532: Skipped 2024-09-22T03:49:17.232 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:49:17.232 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatNS (0 ms) 2024-09-22T03:49:17.232 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.StatRemove 2024-09-22T03:49:17.232 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio.cc:1593: Skipped 2024-09-22T03:49:17.232 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:49:17.232 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.StatRemove (0 ms) 2024-09-22T03:49:17.232 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.ExecuteClass 2024-09-22T03:49:17.232 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio.cc:1649: Skipped 2024-09-22T03:49:17.232 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:49:17.233 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClass (0 ms) 2024-09-22T03:49:17.233 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.MultiWrite 2024-09-22T03:49:17.233 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio.cc:1681: Skipped 2024-09-22T03:49:17.233 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:49:17.233 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.MultiWrite (1 ms) 2024-09-22T03:49:17.233 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 16 tests from LibRadosAioEC (1 ms total) 2024-09-22T03:49:17.233 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:49:17.233 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:49:17.233 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 42 tests from 2 test suites ran. (77207 ms total) 2024-09-22T03:49:17.233 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 26 tests. 2024-09-22T03:49:17.233 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] 16 tests, listed below: 2024-09-22T03:49:17.233 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWrite 2024-09-22T03:49:17.233 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.WaitForComplete 2024-09-22T03:49:17.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip 2024-09-22T03:49:17.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTrip2 2024-09-22T03:49:17.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppend 2024-09-22T03:49:17.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.IsComplete 2024-09-22T03:49:17.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.IsSafe 2024-09-22T03:49:17.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValue 2024-09-22T03:49:17.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.Flush 2024-09-22T03:49:17.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsync 2024-09-22T03:49:17.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFull 2024-09-22T03:49:17.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStat 2024-09-22T03:49:17.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatNS 2024-09-22T03:49:17.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.StatRemove 2024-09-22T03:49:17.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClass 2024-09-22T03:49:17.235 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.MultiWrite 2024-09-22T03:49:17.235 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:49:17.235 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:49:17.235 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_aio_pp 2024-09-22T03:49:17.252 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:49:17.252 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 57 tests from 4 test suites. 2024-09-22T03:49:17.252 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:49:17.253 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 32 tests from LibRadosAio 2024-09-22T03:49:17.253 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.TooBigPP 2024-09-22T03:49:20.185 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.TooBigPP (2932 ms) 2024-09-22T03:49:20.185 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.PoolQuotaPP 2024-09-22T03:49:34.655 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.PoolQuotaPP (14471 ms) 2024-09-22T03:49:34.656 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.SimpleWritePP 2024-09-22T03:49:40.199 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.SimpleWritePP (5544 ms) 2024-09-22T03:49:40.199 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.WaitForSafePP 2024-09-22T03:49:43.217 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.WaitForSafePP (3018 ms) 2024-09-22T03:49:43.217 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RoundTripPP 2024-09-22T03:49:46.237 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RoundTripPP (3019 ms) 2024-09-22T03:49:46.237 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RoundTripPP2 2024-09-22T03:49:49.253 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RoundTripPP2 (3016 ms) 2024-09-22T03:49:49.253 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RoundTripPP3 2024-09-22T03:49:52.288 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RoundTripPP3 (3035 ms) 2024-09-22T03:49:52.289 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-09-22T03:49:55.298 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RoundTripSparseReadPP (3010 ms) 2024-09-22T03:49:55.298 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.IsCompletePP 2024-09-22T03:49:58.192 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.IsCompletePP (2894 ms) 2024-09-22T03:49:58.193 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.IsSafePP 2024-09-22T03:50:01.199 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.IsSafePP (3006 ms) 2024-09-22T03:50:01.199 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.ReturnValuePP 2024-09-22T03:50:04.228 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.ReturnValuePP (3030 ms) 2024-09-22T03:50:04.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.FlushPP 2024-09-22T03:50:07.237 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.FlushPP (3009 ms) 2024-09-22T03:50:07.237 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.FlushAsyncPP 2024-09-22T03:50:10.249 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.FlushAsyncPP (3012 ms) 2024-09-22T03:50:10.249 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-09-22T03:50:13.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RoundTripWriteFullPP (3012 ms) 2024-09-22T03:50:13.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-09-22T03:50:16.282 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RoundTripWriteFullPP2 (3021 ms) 2024-09-22T03:50:16.282 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-09-22T03:50:19.293 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RoundTripWriteSamePP (3011 ms) 2024-09-22T03:50:19.294 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-09-22T03:50:22.319 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RoundTripWriteSamePP2 (3026 ms) 2024-09-22T03:50:22.319 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.SimpleStatPPNS 2024-09-22T03:50:25.333 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.SimpleStatPPNS (3013 ms) 2024-09-22T03:50:25.333 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.SimpleStatPP 2024-09-22T03:50:28.198 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.SimpleStatPP (2866 ms) 2024-09-22T03:50:28.198 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.OperateMtime 2024-09-22T03:50:31.207 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.OperateMtime (3009 ms) 2024-09-22T03:50:31.208 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.OperateMtime2 2024-09-22T03:50:34.221 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.OperateMtime2 (3014 ms) 2024-09-22T03:50:34.222 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.StatRemovePP 2024-09-22T03:50:37.233 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.StatRemovePP (3012 ms) 2024-09-22T03:50:37.233 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.ExecuteClassPP 2024-09-22T03:50:40.253 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.ExecuteClassPP (3019 ms) 2024-09-22T03:50:40.253 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.OmapPP 2024-09-22T03:50:43.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.OmapPP (3010 ms) 2024-09-22T03:50:43.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.MultiWritePP 2024-09-22T03:50:46.285 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.MultiWritePP (3023 ms) 2024-09-22T03:50:46.285 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.AioUnlockPP 2024-09-22T03:50:49.295 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.AioUnlockPP (3010 ms) 2024-09-22T03:50:49.296 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RoundTripAppendPP 2024-09-22T03:50:52.315 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RoundTripAppendPP (3019 ms) 2024-09-22T03:50:52.315 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RacingRemovePP 2024-09-22T03:50:55.326 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RacingRemovePP (3012 ms) 2024-09-22T03:50:55.326 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-09-22T03:50:58.204 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RoundTripCmpExtPP (2878 ms) 2024-09-22T03:50:58.205 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-09-22T03:51:01.218 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.RoundTripCmpExtPP2 (3014 ms) 2024-09-22T03:51:01.218 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.PoolEIOFlag 2024-09-22T03:51:03.734 INFO:tasks.workunit.client.0.smithi003.stdout:setting pool EIO 2024-09-22T03:51:13.398 INFO:tasks.workunit.client.0.smithi003.stdout:waiting for 812 2024-09-22T03:51:14.398 INFO:tasks.workunit.client.0.smithi003.stdout:waiting for 771 2024-09-22T03:51:15.398 INFO:tasks.workunit.client.0.smithi003.stdout:waiting for 729 2024-09-22T03:51:16.398 INFO:tasks.workunit.client.0.smithi003.stdout:waiting for 653 2024-09-22T03:51:17.398 INFO:tasks.workunit.client.0.smithi003.stdout:waiting for 572 2024-09-22T03:51:18.398 INFO:tasks.workunit.client.0.smithi003.stdout:waiting for 487 2024-09-22T03:51:19.398 INFO:tasks.workunit.client.0.smithi003.stdout:waiting for 403 2024-09-22T03:51:20.398 INFO:tasks.workunit.client.0.smithi003.stdout:waiting for 323 2024-09-22T03:51:21.399 INFO:tasks.workunit.client.0.smithi003.stdout:max_success 750, min_failed 751 2024-09-22T03:51:21.679 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAio.PoolEIOFlag (20460 ms) 2024-09-22T03:51:21.679 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAio.MultiReads 2024-09-22T03:51:21.679 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:2427: Skipped 2024-09-22T03:51:21.679 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:21.679 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAio.MultiReads (0 ms) 2024-09-22T03:51:21.679 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 32 tests from LibRadosAio (124426 ms total) 2024-09-22T03:51:21.679 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:51:21.679 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosAioPP 2024-09-22T03:51:21.679 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-09-22T03:51:29.722 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAioPP.ReadIntoBufferlist (8044 ms) 2024-09-22T03:51:29.723 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-09-22T03:51:38.228 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAioPP.XattrsRoundTripPP (8505 ms) 2024-09-22T03:51:38.228 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioPP.RmXattrPP 2024-09-22T03:51:53.216 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAioPP.RmXattrPP (14989 ms) 2024-09-22T03:51:53.216 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioPP.RemoveTestPP 2024-09-22T03:51:56.231 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosAioPP.RemoveTestPP (3015 ms) 2024-09-22T03:51:56.232 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosAioPP (34553 ms total) 2024-09-22T03:51:56.232 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:51:56.232 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibRadosIoPP 2024-09-22T03:51:56.232 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.XattrListPP 2024-09-22T03:51:59.254 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.XattrListPP (3019 ms) 2024-09-22T03:51:59.254 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibRadosIoPP (3019 ms total) 2024-09-22T03:51:59.254 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:51:59.255 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 20 tests from LibRadosAioEC 2024-09-22T03:51:59.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.SimpleWritePP 2024-09-22T03:51:59.255 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:1384: Skipped 2024-09-22T03:51:59.255 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWritePP (0 ms) 2024-09-22T03:51:59.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.WaitForSafePP 2024-09-22T03:51:59.255 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:1419: Skipped 2024-09-22T03:51:59.255 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.WaitForSafePP (0 ms) 2024-09-22T03:51:59.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.RoundTripPP 2024-09-22T03:51:59.255 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:1436: Skipped 2024-09-22T03:51:59.255 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP (0 ms) 2024-09-22T03:51:59.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.RoundTripPP2 2024-09-22T03:51:59.256 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:1467: Skipped 2024-09-22T03:51:59.256 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP2 (0 ms) 2024-09-22T03:51:59.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.RoundTripPP3 2024-09-22T03:51:59.256 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:1500: Skipped 2024-09-22T03:51:59.256 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP3 (0 ms) 2024-09-22T03:51:59.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-09-22T03:51:59.256 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:1605: Skipped 2024-09-22T03:51:59.256 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripSparseReadPP (0 ms) 2024-09-22T03:51:59.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-09-22T03:51:59.257 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:1637: Skipped 2024-09-22T03:51:59.257 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.257 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppendPP (0 ms) 2024-09-22T03:51:59.257 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.IsCompletePP 2024-09-22T03:51:59.257 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:1705: Skipped 2024-09-22T03:51:59.257 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.257 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.IsCompletePP (0 ms) 2024-09-22T03:51:59.257 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.IsSafePP 2024-09-22T03:51:59.257 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:1742: Skipped 2024-09-22T03:51:59.257 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.257 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.IsSafePP (0 ms) 2024-09-22T03:51:59.257 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.ReturnValuePP 2024-09-22T03:51:59.257 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:1780: Skipped 2024-09-22T03:51:59.257 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.257 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValuePP (0 ms) 2024-09-22T03:51:59.258 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.FlushPP 2024-09-22T03:51:59.258 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:1796: Skipped 2024-09-22T03:51:59.258 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.258 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.FlushPP (0 ms) 2024-09-22T03:51:59.258 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.FlushAsyncPP 2024-09-22T03:51:59.258 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:1824: Skipped 2024-09-22T03:51:59.258 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.258 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsyncPP (0 ms) 2024-09-22T03:51:59.258 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-09-22T03:51:59.258 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:1859: Skipped 2024-09-22T03:51:59.258 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.258 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP (0 ms) 2024-09-22T03:51:59.258 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-09-22T03:51:59.259 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:1904: Skipped 2024-09-22T03:51:59.259 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP2 (0 ms) 2024-09-22T03:51:59.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.SimpleStatPP 2024-09-22T03:51:59.259 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:1946: Skipped 2024-09-22T03:51:59.259 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPP (0 ms) 2024-09-22T03:51:59.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-09-22T03:51:59.259 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:1977: Skipped 2024-09-22T03:51:59.259 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPPNS (0 ms) 2024-09-22T03:51:59.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.StatRemovePP 2024-09-22T03:51:59.259 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:2008: Skipped 2024-09-22T03:51:59.260 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.StatRemovePP (0 ms) 2024-09-22T03:51:59.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.ExecuteClassPP 2024-09-22T03:51:59.260 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:2059: Skipped 2024-09-22T03:51:59.260 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClassPP (0 ms) 2024-09-22T03:51:59.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.OmapPP 2024-09-22T03:51:59.260 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:2089: Skipped 2024-09-22T03:51:59.260 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.OmapPP (0 ms) 2024-09-22T03:51:59.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosAioEC.MultiWritePP 2024-09-22T03:51:59.260 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/aio_cxx.cc:2124: Skipped 2024-09-22T03:51:59.261 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:51:59.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.MultiWritePP (0 ms) 2024-09-22T03:51:59.261 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 20 tests from LibRadosAioEC (0 ms total) 2024-09-22T03:51:59.261 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:51:59.261 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:51:59.261 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 57 tests from 4 test suites ran. (161999 ms total) 2024-09-22T03:51:59.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 36 tests. 2024-09-22T03:51:59.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] 21 tests, listed below: 2024-09-22T03:51:59.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAio.MultiReads 2024-09-22T03:51:59.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.SimpleWritePP 2024-09-22T03:51:59.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.WaitForSafePP 2024-09-22T03:51:59.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP 2024-09-22T03:51:59.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP2 2024-09-22T03:51:59.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripPP3 2024-09-22T03:51:59.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripSparseReadPP 2024-09-22T03:51:59.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripAppendPP 2024-09-22T03:51:59.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.IsCompletePP 2024-09-22T03:51:59.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.IsSafePP 2024-09-22T03:51:59.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.ReturnValuePP 2024-09-22T03:51:59.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.FlushPP 2024-09-22T03:51:59.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.FlushAsyncPP 2024-09-22T03:51:59.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP 2024-09-22T03:51:59.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.RoundTripWriteFullPP2 2024-09-22T03:51:59.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPP 2024-09-22T03:51:59.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.SimpleStatPPNS 2024-09-22T03:51:59.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.StatRemovePP 2024-09-22T03:51:59.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.ExecuteClassPP 2024-09-22T03:51:59.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.OmapPP 2024-09-22T03:51:59.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosAioEC.MultiWritePP 2024-09-22T03:51:59.263 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:51:59.263 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:51:59.263 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_io 2024-09-22T03:51:59.276 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:51:59.277 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 24 tests from 2 test suites. 2024-09-22T03:51:59.277 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:51:59.277 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 14 tests from LibRadosIo 2024-09-22T03:52:01.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIo.SimpleWrite 2024-09-22T03:52:01.728 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIo.SimpleWrite (468 ms) 2024-09-22T03:52:01.729 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIo.TooBig 2024-09-22T03:52:01.730 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIo.TooBig (0 ms) 2024-09-22T03:52:01.730 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIo.ReadTimeout 2024-09-22T03:52:01.821 INFO:tasks.workunit.client.0.smithi003.stdout:no timeout :/ 2024-09-22T03:52:01.821 INFO:tasks.workunit.client.0.smithi003.stdout:no timeout :/ 2024-09-22T03:52:01.821 INFO:tasks.workunit.client.0.smithi003.stdout:no timeout :/ 2024-09-22T03:52:01.821 INFO:tasks.workunit.client.0.smithi003.stdout:no timeout :/ 2024-09-22T03:52:01.821 INFO:tasks.workunit.client.0.smithi003.stdout:no timeout :/ 2024-09-22T03:52:01.821 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIo.ReadTimeout (93 ms) 2024-09-22T03:52:01.822 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIo.RoundTrip 2024-09-22T03:52:01.886 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIo.RoundTrip (66 ms) 2024-09-22T03:52:01.886 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIo.Checksum 2024-09-22T03:52:01.917 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIo.Checksum (30 ms) 2024-09-22T03:52:01.917 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-09-22T03:52:01.970 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIo.OverlappingWriteRoundTrip (53 ms) 2024-09-22T03:52:01.970 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIo.WriteFullRoundTrip 2024-09-22T03:52:02.030 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIo.WriteFullRoundTrip (61 ms) 2024-09-22T03:52:02.030 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIo.AppendRoundTrip 2024-09-22T03:52:02.079 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIo.AppendRoundTrip (49 ms) 2024-09-22T03:52:02.079 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIo.ZeroLenZero 2024-09-22T03:52:02.085 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIo.ZeroLenZero (6 ms) 2024-09-22T03:52:02.085 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIo.TruncTest 2024-09-22T03:52:02.137 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIo.TruncTest (51 ms) 2024-09-22T03:52:02.137 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIo.RemoveTest 2024-09-22T03:52:02.177 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIo.RemoveTest (41 ms) 2024-09-22T03:52:02.178 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIo.XattrsRoundTrip 2024-09-22T03:52:02.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIo.XattrsRoundTrip (52 ms) 2024-09-22T03:52:02.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIo.RmXattr 2024-09-22T03:52:02.387 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIo.RmXattr (158 ms) 2024-09-22T03:52:02.387 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIo.XattrIter 2024-09-22T03:52:02.448 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIo.XattrIter (60 ms) 2024-09-22T03:52:03.221 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 14 tests from LibRadosIo (1188 ms total) 2024-09-22T03:52:03.221 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:52:03.221 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 10 tests from LibRadosIoEC 2024-09-22T03:52:03.221 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:168: Skipped 2024-09-22T03:52:03.221 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.221 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoEC.SimpleWrite 2024-09-22T03:52:03.221 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:03.221 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.221 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:03.221 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.221 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.SimpleWrite (0 ms) 2024-09-22T03:52:03.222 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoEC.RoundTrip 2024-09-22T03:52:03.222 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:03.222 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.222 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:03.222 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.222 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.RoundTrip (0 ms) 2024-09-22T03:52:03.222 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-09-22T03:52:03.222 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:03.222 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.222 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:03.222 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.222 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.OverlappingWriteRoundTrip (0 ms) 2024-09-22T03:52:03.222 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-09-22T03:52:03.223 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:03.223 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.223 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:03.223 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.223 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.WriteFullRoundTrip (0 ms) 2024-09-22T03:52:03.223 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoEC.AppendRoundTrip 2024-09-22T03:52:03.223 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:03.223 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.223 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:03.223 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.223 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.AppendRoundTrip (0 ms) 2024-09-22T03:52:03.223 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoEC.TruncTest 2024-09-22T03:52:03.224 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:03.224 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.224 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:03.224 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.224 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.TruncTest (0 ms) 2024-09-22T03:52:03.224 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoEC.RemoveTest 2024-09-22T03:52:03.224 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:03.224 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.224 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:03.224 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.224 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.RemoveTest (0 ms) 2024-09-22T03:52:03.224 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-09-22T03:52:03.224 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:03.224 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.225 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:03.225 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.225 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.XattrsRoundTrip (0 ms) 2024-09-22T03:52:03.225 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoEC.RmXattr 2024-09-22T03:52:03.225 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:03.225 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.225 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:03.225 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.225 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.RmXattr (0 ms) 2024-09-22T03:52:03.225 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoEC.XattrIter 2024-09-22T03:52:03.225 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:03.225 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.225 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:03.225 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.225 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.XattrIter (0 ms) 2024-09-22T03:52:03.226 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:176: Skipped 2024-09-22T03:52:03.226 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:03.226 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 10 tests from LibRadosIoEC (0 ms total) 2024-09-22T03:52:03.226 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:52:03.226 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:52:03.226 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 24 tests from 2 test suites ran. (3943 ms total) 2024-09-22T03:52:03.226 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 14 tests. 2024-09-22T03:52:03.226 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] 10 tests, listed below: 2024-09-22T03:52:03.226 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.SimpleWrite 2024-09-22T03:52:03.226 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.RoundTrip 2024-09-22T03:52:03.226 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-09-22T03:52:03.227 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.WriteFullRoundTrip 2024-09-22T03:52:03.227 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.AppendRoundTrip 2024-09-22T03:52:03.227 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.TruncTest 2024-09-22T03:52:03.227 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.RemoveTest 2024-09-22T03:52:03.227 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.XattrsRoundTrip 2024-09-22T03:52:03.227 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.RmXattr 2024-09-22T03:52:03.227 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoEC.XattrIter 2024-09-22T03:52:03.228 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:52:03.228 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:52:03.228 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_io_pp 2024-09-22T03:52:03.246 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:52:03.246 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 40 tests from 2 test suites. 2024-09-22T03:52:03.246 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:52:03.246 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 22 tests from LibRadosIoPP 2024-09-22T03:52:03.246 INFO:tasks.workunit.client.0.smithi003.stdout:seed 30921 2024-09-22T03:52:05.277 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.TooBigPP 2024-09-22T03:52:05.277 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.TooBigPP (1 ms) 2024-09-22T03:52:05.277 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.SimpleWritePP 2024-09-22T03:52:05.777 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.SimpleWritePP (499 ms) 2024-09-22T03:52:05.777 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.ReadOpPP 2024-09-22T03:52:05.885 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.ReadOpPP (109 ms) 2024-09-22T03:52:05.886 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.SparseReadOpPP 2024-09-22T03:52:05.928 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.SparseReadOpPP (43 ms) 2024-09-22T03:52:05.928 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.SparseReadExtentArrayOpPP 2024-09-22T03:52:10.333 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.SparseReadExtentArrayOpPP (4404 ms) 2024-09-22T03:52:10.333 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.RoundTripPP 2024-09-22T03:52:10.370 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.RoundTripPP (37 ms) 2024-09-22T03:52:10.370 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.RoundTripPP2 2024-09-22T03:52:10.402 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.RoundTripPP2 (32 ms) 2024-09-22T03:52:10.402 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.Checksum 2024-09-22T03:52:10.434 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.Checksum (32 ms) 2024-09-22T03:52:10.434 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-09-22T03:52:10.464 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.ReadIntoBufferlist (30 ms) 2024-09-22T03:52:10.465 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-09-22T03:52:10.513 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (49 ms) 2024-09-22T03:52:10.514 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-09-22T03:52:10.568 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.WriteFullRoundTripPP (55 ms) 2024-09-22T03:52:10.569 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-09-22T03:52:10.599 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (30 ms) 2024-09-22T03:52:10.599 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-09-22T03:52:10.650 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.AppendRoundTripPP (51 ms) 2024-09-22T03:52:10.650 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.TruncTestPP 2024-09-22T03:52:10.700 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.TruncTestPP (51 ms) 2024-09-22T03:52:10.700 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.RemoveTestPP 2024-09-22T03:52:10.747 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.RemoveTestPP (46 ms) 2024-09-22T03:52:10.747 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-09-22T03:52:10.798 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.XattrsRoundTripPP (52 ms) 2024-09-22T03:52:10.799 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.RmXattrPP 2024-09-22T03:52:10.927 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.RmXattrPP (129 ms) 2024-09-22T03:52:10.927 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.XattrListPP 2024-09-22T03:52:10.992 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.XattrListPP (64 ms) 2024-09-22T03:52:10.992 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.CrcZeroWrite 2024-09-22T03:52:11.036 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.CrcZeroWrite (45 ms) 2024-09-22T03:52:11.036 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.CmpExtPP 2024-09-22T03:52:11.088 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.CmpExtPP (52 ms) 2024-09-22T03:52:11.089 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-09-22T03:52:11.118 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.CmpExtDNEPP (30 ms) 2024-09-22T03:52:11.118 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-09-22T03:52:11.160 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosIoPP.CmpExtMismatchPP (41 ms) 2024-09-22T03:52:11.225 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 22 tests from LibRadosIoPP (5883 ms total) 2024-09-22T03:52:11.225 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:52:11.225 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 18 tests from LibRadosIoECPP 2024-09-22T03:52:11.225 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:375: Skipped 2024-09-22T03:52:11.225 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.225 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.SimpleWritePP 2024-09-22T03:52:11.225 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.225 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.225 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.225 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.226 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.SimpleWritePP (0 ms) 2024-09-22T03:52:11.226 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.ReadOpPP 2024-09-22T03:52:11.226 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.226 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.226 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.226 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.226 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.ReadOpPP (0 ms) 2024-09-22T03:52:11.226 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-09-22T03:52:11.226 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.226 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.226 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.226 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.226 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.SparseReadOpPP (0 ms) 2024-09-22T03:52:11.227 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.RoundTripPP 2024-09-22T03:52:11.227 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.227 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.227 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.227 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.227 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP (0 ms) 2024-09-22T03:52:11.227 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.RoundTripPP2 2024-09-22T03:52:11.227 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.227 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.227 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.227 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.227 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP2 (0 ms) 2024-09-22T03:52:11.227 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-09-22T03:52:11.227 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.228 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.228 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.228 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.228 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.OverlappingWriteRoundTripPP (0 ms) 2024-09-22T03:52:11.228 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-09-22T03:52:11.228 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.228 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.228 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.228 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.228 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP (0 ms) 2024-09-22T03:52:11.228 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-09-22T03:52:11.228 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.228 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.228 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.229 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP2 (0 ms) 2024-09-22T03:52:11.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-09-22T03:52:11.229 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.229 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.229 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.229 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.AppendRoundTripPP (0 ms) 2024-09-22T03:52:11.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.TruncTestPP 2024-09-22T03:52:11.229 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.229 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.230 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.230 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.230 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.TruncTestPP (0 ms) 2024-09-22T03:52:11.230 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.RemoveTestPP 2024-09-22T03:52:11.230 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.230 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.230 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.230 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.230 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.RemoveTestPP (0 ms) 2024-09-22T03:52:11.230 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-09-22T03:52:11.231 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.231 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.231 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.231 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.231 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.XattrsRoundTripPP (0 ms) 2024-09-22T03:52:11.231 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.RmXattrPP 2024-09-22T03:52:11.231 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.231 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.231 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.231 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.231 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.RmXattrPP (0 ms) 2024-09-22T03:52:11.231 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.CrcZeroWrite 2024-09-22T03:52:11.231 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.231 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.232 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.232 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.232 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.CrcZeroWrite (0 ms) 2024-09-22T03:52:11.232 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.XattrListPP 2024-09-22T03:52:11.232 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.232 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.232 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.232 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.232 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.XattrListPP (0 ms) 2024-09-22T03:52:11.232 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.CmpExtPP 2024-09-22T03:52:11.232 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.232 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.232 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.232 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.232 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtPP (0 ms) 2024-09-22T03:52:11.233 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-09-22T03:52:11.233 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.233 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.233 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.233 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.233 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtDNEPP (0 ms) 2024-09-22T03:52:11.233 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-09-22T03:52:11.233 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:11.233 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.233 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:11.234 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtMismatchPP (0 ms) 2024-09-22T03:52:11.234 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:383: Skipped 2024-09-22T03:52:11.234 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:11.234 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 18 tests from LibRadosIoECPP (0 ms total) 2024-09-22T03:52:11.234 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:52:11.234 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:52:11.234 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 40 tests from 2 test suites ran. (7975 ms total) 2024-09-22T03:52:11.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 22 tests. 2024-09-22T03:52:11.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] 18 tests, listed below: 2024-09-22T03:52:11.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.SimpleWritePP 2024-09-22T03:52:11.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.ReadOpPP 2024-09-22T03:52:11.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.SparseReadOpPP 2024-09-22T03:52:11.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP 2024-09-22T03:52:11.234 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.RoundTripPP2 2024-09-22T03:52:11.235 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-09-22T03:52:11.235 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP 2024-09-22T03:52:11.235 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-09-22T03:52:11.235 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.AppendRoundTripPP 2024-09-22T03:52:11.235 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.TruncTestPP 2024-09-22T03:52:11.235 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.RemoveTestPP 2024-09-22T03:52:11.235 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.XattrsRoundTripPP 2024-09-22T03:52:11.235 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.RmXattrPP 2024-09-22T03:52:11.235 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.CrcZeroWrite 2024-09-22T03:52:11.235 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.XattrListPP 2024-09-22T03:52:11.235 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtPP 2024-09-22T03:52:11.235 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtDNEPP 2024-09-22T03:52:11.235 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosIoECPP.CmpExtMismatchPP 2024-09-22T03:52:11.236 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:52:11.236 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:52:11.236 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_asio 2024-09-22T03:52:11.257 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.256+0000 7f42546df6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:52:11.264 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.263+0000 7f42546df6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:52:11.264 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.263+0000 7f42546df6c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:52:11.265 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 12 tests from 1 test suite. 2024-09-22T03:52:11.265 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:52:11.265 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 12 tests from AsioRados 2024-09-22T03:52:11.266 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.265+0000 7f42546df6c0 1 Processor -- start 2024-09-22T03:52:11.266 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.265+0000 7f42546df6c0 1 -- start start 2024-09-22T03:52:11.269 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.267+0000 7f42546df6c0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x21b2ba0 0x22b6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:52:11.269 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.267+0000 7f42546df6c0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x2117520 con 0x21b2ba0 2024-09-22T03:52:11.269 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.267+0000 7f424d03f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x21b2ba0 0x22b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:52:11.269 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.268+0000 7f424d03f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x21b2ba0 0x22b6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41182/0 (socket says 172.21.15.3:41182) 2024-09-22T03:52:11.270 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.268+0000 7f424d03f640 1 -- 172.21.15.3:0/3275083156 learned_addr learned my addr 172.21.15.3:0/3275083156 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:52:11.270 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.268+0000 7f424d03f640 1 -- 172.21.15.3:0/3275083156 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x20b3e10 con 0x21b2ba0 2024-09-22T03:52:11.270 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.268+0000 7f424d03f640 1 --2- 172.21.15.3:0/3275083156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x21b2ba0 0x22b6070 crc :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93aa749868f22098 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:52:11.270 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.269+0000 7f4246ffd640 1 -- 172.21.15.3:0/3275083156 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f423802ee00 con 0x21b2ba0 2024-09-22T03:52:11.270 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.269+0000 7f4246ffd640 1 -- 172.21.15.3:0/3275083156 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4238034040 con 0x21b2ba0 2024-09-22T03:52:11.270 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.269+0000 7f4246ffd640 1 -- 172.21.15.3:0/3275083156 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f42380373d0 con 0x21b2ba0 2024-09-22T03:52:11.272 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.270+0000 7f42546df6c0 1 -- 172.21.15.3:0/3275083156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x21b2ba0 msgr2=0x22b6070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:52:11.272 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.270+0000 7f42546df6c0 1 --2- 172.21.15.3:0/3275083156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x21b2ba0 0x22b6070 crc :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:52:11.272 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.270+0000 7f42546df6c0 1 -- 172.21.15.3:0/3275083156 shutdown_connections 2024-09-22T03:52:11.272 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.270+0000 7f42546df6c0 1 --2- 172.21.15.3:0/3275083156 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x21b2ba0 0x22b6070 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:52:11.272 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.270+0000 7f42546df6c0 1 -- 172.21.15.3:0/3275083156 >> 172.21.15.3:0/3275083156 conn(0x20d7770 msgr2=0x21ad9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:52:11.272 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.270+0000 7f42546df6c0 1 -- 172.21.15.3:0/3275083156 shutdown_connections 2024-09-22T03:52:11.272 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.270+0000 7f42546df6c0 1 -- 172.21.15.3:0/3275083156 wait complete. 2024-09-22T03:52:11.273 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.272+0000 7f42546df6c0 1 Processor -- start 2024-09-22T03:52:11.273 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.272+0000 7f42546df6c0 1 -- start start 2024-09-22T03:52:11.276 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.274+0000 7f42546df6c0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x21b2ba0 0x22ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:52:11.276 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.275+0000 7f42546df6c0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x2115280 con 0x21b2ba0 2024-09-22T03:52:11.276 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.275+0000 7f424d03f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x21b2ba0 0x22ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:52:11.276 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.275+0000 7f424d03f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x21b2ba0 0x22ccb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41192/0 (socket says 172.21.15.3:41192) 2024-09-22T03:52:11.276 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.275+0000 7f424d03f640 1 -- 172.21.15.3:0/3755633323 learned_addr learned my addr 172.21.15.3:0/3755633323 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T03:52:11.277 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.275+0000 7f424d03f640 1 -- 172.21.15.3:0/3755633323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x20a5920 con 0x21b2ba0 2024-09-22T03:52:11.277 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.276+0000 7f424d03f640 1 --2- 172.21.15.3:0/3755633323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x21b2ba0 0x22ccb70 crc :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:52:11.277 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.276+0000 7f4245ffb640 1 -- 172.21.15.3:0/3755633323 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f423803bb10 con 0x21b2ba0 2024-09-22T03:52:11.277 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.276+0000 7f4245ffb640 1 -- 172.21.15.3:0/3755633323 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f423803bc70 con 0x21b2ba0 2024-09-22T03:52:11.277 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.276+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x20a5380 con 0x21b2ba0 2024-09-22T03:52:11.279 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.277+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x22d0dc0 con 0x21b2ba0 2024-09-22T03:52:11.279 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.277+0000 7f4245ffb640 1 -- 172.21.15.3:0/3755633323 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f423803c410 con 0x21b2ba0 2024-09-22T03:52:11.280 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.279+0000 7f4245ffb640 1 -- 172.21.15.3:0/3755633323 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f423803c5c0 con 0x21b2ba0 2024-09-22T03:52:11.282 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.280+0000 7f4245ffb640 1 --2- 172.21.15.3:0/3755633323 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f422803f4e0 0x7f42280419f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:52:11.282 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.280+0000 7f4245ffb640 1 -- 172.21.15.3:0/3755633323 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(239..239 src has 1..239) ==== 2700+0+0 (crc 0 0 0) 0x7f4238033040 con 0x21b2ba0 2024-09-22T03:52:11.282 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.280+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x210ddf0 con 0x21b2ba0 2024-09-22T03:52:11.282 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.280+0000 7f424c83e640 1 --2- 172.21.15.3:0/3755633323 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f422803f4e0 0x7f42280419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T03:52:11.282 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.281+0000 7f4245ffb640 1 -- 172.21.15.3:0/3755633323 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=239) ==== 24+0+0 (crc 0 0 0) 0x7f4238039710 con 0x21b2ba0 2024-09-22T03:52:11.282 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.281+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- pool_op(create pool 0 tid 1 name ceph_test_rados_api_asio v0) -- 0x2120bf0 con 0x21b2ba0 2024-09-22T03:52:11.283 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:11.281+0000 7f424c83e640 1 --2- 172.21.15.3:0/3755633323 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f422803f4e0 0x7f42280419f0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:52:12.227 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:12.225+0000 7f4245ffb640 1 -- 172.21.15.3:0/3755633323 <== mon.0 v2:172.21.15.3:3300/0 7 ==== pool_op_reply(tid 1 (0) Success v240) ==== 43+0+0 (crc 0 0 0) 0x7f4238036e20 con 0x21b2ba0 2024-09-22T03:52:12.227 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:12.225+0000 7f4245ffb640 1 -- 172.21.15.3:0/3755633323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=240}) -- 0x7f4228043e60 con 0x21b2ba0 2024-09-22T03:52:12.227 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:12.226+0000 7f4245ffb640 1 -- 172.21.15.3:0/3755633323 <== mon.0 v2:172.21.15.3:3300/0 8 ==== osd_map(240..240 src has 1..240) ==== 651+0+0 (crc 0 0 0) 0x7f4238007e30 con 0x21b2ba0 2024-09-22T03:52:12.227 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:12.226+0000 7f42546df6c0 1 --2- 172.21.15.3:0/3755633323 >> v2:172.21.15.3:6802/2784566484 conn(0x21b2fb0 0x22e0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T03:52:12.228 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:12.226+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:2 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e240) -- 0x22d4270 con 0x21b2fb0 2024-09-22T03:52:12.230 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:12.229+0000 7f424d840640 1 --2- 172.21.15.3:0/3755633323 >> v2:172.21.15.3:6802/2784566484 conn(0x21b2fb0 0x22e0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T03:52:12.244 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:12.243+0000 7f424d840640 1 --2- 172.21.15.3:0/3755633323 >> v2:172.21.15.3:6802/2784566484 conn(0x21b2fb0 0x22e0190 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T03:52:13.568 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.567+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 1 ==== osd_op_reply(2 exist [writefull 0~5] v241'1 uv1 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.568 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] AsioRados.AsyncReadCallback 2024-09-22T03:52:13.569 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.567+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:3 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e240) -- 0x22ce010 con 0x21b2fb0 2024-09-22T03:52:13.569 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.567+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:4 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e240) -- 0x22ce870 con 0x21b2fb0 2024-09-22T03:52:13.581 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.580+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 2 ==== osd_op_reply(4 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.583 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.582+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 3 ==== osd_op_reply(3 exist [read 0~5 out=5b] v241'1 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.583 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] AsioRados.AsyncReadCallback (15 ms) 2024-09-22T03:52:13.583 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] AsioRados.AsyncReadFuture 2024-09-22T03:52:13.583 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.582+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:5 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e240) -- 0x22cec50 con 0x21b2fb0 2024-09-22T03:52:13.583 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.582+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:6 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e240) -- 0x22e06d0 con 0x21b2fb0 2024-09-22T03:52:13.596 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.594+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 4 ==== osd_op_reply(6 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.596 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.595+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 5 ==== osd_op_reply(5 exist [read 0~5 out=5b] v241'1 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.596 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] AsioRados.AsyncReadFuture (13 ms) 2024-09-22T03:52:13.596 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] AsioRados.AsyncReadYield 2024-09-22T03:52:13.597 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.595+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:7 75.1 75:8049a5cb:::exist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e240) -- 0x22ce3c0 con 0x21b2fb0 2024-09-22T03:52:13.597 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.595+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:8 75.1 75:855dde50:::noexist:head [read 0~256] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e240) -- 0x2321050 con 0x21b2fb0 2024-09-22T03:52:13.608 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.607+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 6 ==== osd_op_reply(8 noexist [read 0~256] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.608 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.607+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 7 ==== osd_op_reply(7 exist [read 0~5 out=5b] v241'1 uv1 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.609 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] AsioRados.AsyncReadYield (12 ms) 2024-09-22T03:52:13.609 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] AsioRados.AsyncWriteCallback 2024-09-22T03:52:13.609 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.607+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:9 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e240) -- 0x22cd200 con 0x21b2fb0 2024-09-22T03:52:13.654 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.652+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 8 ==== osd_op_reply(9 exist [write 0~5] v241'2 uv2 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.654 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] AsioRados.AsyncWriteCallback (45 ms) 2024-09-22T03:52:13.654 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] AsioRados.AsyncWriteFuture 2024-09-22T03:52:13.654 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.652+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:10 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e240) -- 0x22cd200 con 0x21b2fb0 2024-09-22T03:52:13.684 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.683+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 9 ==== osd_op_reply(10 exist [write 0~5] v241'3 uv3 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.684 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] AsioRados.AsyncWriteFuture (31 ms) 2024-09-22T03:52:13.684 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] AsioRados.AsyncWriteYield 2024-09-22T03:52:13.684 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.683+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:11 75.1 75:8049a5cb:::exist:head [write 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e240) -- 0x2303840 con 0x21b2fb0 2024-09-22T03:52:13.706 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.705+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 10 ==== osd_op_reply(11 exist [write 0~5] v241'4 uv4 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.707 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] AsioRados.AsyncWriteYield (22 ms) 2024-09-22T03:52:13.707 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] AsioRados.AsyncReadOperationCallback 2024-09-22T03:52:13.707 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.705+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:12 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e240) -- 0x22e0a30 con 0x21b2fb0 2024-09-22T03:52:13.707 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.705+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:13 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e240) -- 0x22e1390 con 0x21b2fb0 2024-09-22T03:52:13.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.713+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 11 ==== osd_op_reply(13 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.713+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 12 ==== osd_op_reply(12 exist [read 0~5 out=5b] v241'4 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.715 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] AsioRados.AsyncReadOperationCallback (8 ms) 2024-09-22T03:52:13.715 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] AsioRados.AsyncReadOperationFuture 2024-09-22T03:52:13.715 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.714+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:14 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e240) -- 0x22e0ee0 con 0x21b2fb0 2024-09-22T03:52:13.715 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.714+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:15 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e240) -- 0x2303de0 con 0x21b2fb0 2024-09-22T03:52:13.723 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.722+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 13 ==== osd_op_reply(15 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.723 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.722+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 14 ==== osd_op_reply(14 exist [read 0~5 out=5b] v241'4 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.723 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] AsioRados.AsyncReadOperationFuture (9 ms) 2024-09-22T03:52:13.723 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] AsioRados.AsyncReadOperationYield 2024-09-22T03:52:13.724 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.722+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:16 75.1 75:8049a5cb:::exist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e240) -- 0x2303840 con 0x21b2fb0 2024-09-22T03:52:13.724 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.722+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:17 75.1 75:855dde50:::noexist:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e240) -- 0x2304190 con 0x21b2fb0 2024-09-22T03:52:13.731 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.730+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 15 ==== osd_op_reply(17 noexist [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.731 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.730+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 16 ==== osd_op_reply(16 exist [read 0~5 out=5b] v241'4 uv4 ondisk = 0) ==== 149+0+5 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.732 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] AsioRados.AsyncReadOperationYield (8 ms) 2024-09-22T03:52:13.732 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] AsioRados.AsyncWriteOperationCallback 2024-09-22T03:52:13.732 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.730+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:18 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e240) -- 0x22e14c0 con 0x21b2fb0 2024-09-22T03:52:13.750 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.749+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 17 ==== osd_op_reply(18 exist [writefull 0~5] v241'5 uv5 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.750 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] AsioRados.AsyncWriteOperationCallback (19 ms) 2024-09-22T03:52:13.751 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] AsioRados.AsyncWriteOperationFuture 2024-09-22T03:52:13.751 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.749+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:19 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e240) -- 0x22e14c0 con 0x21b2fb0 2024-09-22T03:52:13.770 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.769+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 18 ==== osd_op_reply(19 exist [writefull 0~5] v241'6 uv6 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.770 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] AsioRados.AsyncWriteOperationFuture (20 ms) 2024-09-22T03:52:13.770 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] AsioRados.AsyncWriteOperationYield 2024-09-22T03:52:13.771 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.769+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:20 75.1 75:8049a5cb:::exist:head [writefull 0~5 in=5b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e240) -- 0x2303840 con 0x21b2fb0 2024-09-22T03:52:13.795 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.794+0000 7f424d840640 1 -- 172.21.15.3:0/3755633323 <== osd.1 v2:172.21.15.3:6802/2784566484 19 ==== osd_op_reply(20 exist [writefull 0~5] v241'7 uv7 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f424000c430 con 0x21b2fb0 2024-09-22T03:52:13.795 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] AsioRados.AsyncWriteOperationYield (25 ms) 2024-09-22T03:52:13.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.794+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 >> v2:172.21.15.3:6802/2784566484 conn(0x21b2fb0 msgr2=0x22e0190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:52:13.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.794+0000 7f42546df6c0 1 --2- 172.21.15.3:0/3755633323 >> v2:172.21.15.3:6802/2784566484 conn(0x21b2fb0 0x22e0190 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:52:13.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.794+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f422803f4e0 msgr2=0x7f42280419f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:52:13.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.794+0000 7f42546df6c0 1 --2- 172.21.15.3:0/3755633323 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f422803f4e0 0x7f42280419f0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:52:13.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.794+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x21b2ba0 msgr2=0x22ccb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T03:52:13.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.794+0000 7f42546df6c0 1 --2- 172.21.15.3:0/3755633323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x21b2ba0 0x22ccb70 crc :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:52:13.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.794+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 shutdown_connections 2024-09-22T03:52:13.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.794+0000 7f42546df6c0 1 --2- 172.21.15.3:0/3755633323 >> v2:172.21.15.3:6802/2784566484 conn(0x21b2fb0 0x22e0190 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:52:13.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.794+0000 7f42546df6c0 1 --2- 172.21.15.3:0/3755633323 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f422803f4e0 0x7f42280419f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:52:13.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.794+0000 7f42546df6c0 1 --2- 172.21.15.3:0/3755633323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x21b2ba0 0x22ccb70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T03:52:13.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.794+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 >> 172.21.15.3:0/3755633323 conn(0x20d7770 msgr2=0x21b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T03:52:13.798 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.794+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 shutdown_connections 2024-09-22T03:52:13.798 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.794+0000 7f42546df6c0 1 -- 172.21.15.3:0/3755633323 wait complete. 2024-09-22T03:52:13.798 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 12 tests from AsioRados (227 ms total) 2024-09-22T03:52:13.799 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:52:13.799 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:52:13.799 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 12 tests from 1 test suite ran. (2531 ms total) 2024-09-22T03:52:13.799 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 12 tests. 2024-09-22T03:52:13.800 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:52:13.800 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:52:13.800 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_list 2024-09-22T03:52:13.829 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.828+0000 7f70aba349c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:52:13.829 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.828+0000 7f70aba349c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:52:13.829 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:13.828+0000 7f70aba349c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:52:13.830 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 11 tests from 3 test suites. 2024-09-22T03:52:13.830 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:52:13.830 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 7 tests from LibRadosList 2024-09-22T03:52:15.242 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosList.ListObjects 2024-09-22T03:52:16.007 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosList.ListObjects (764 ms) 2024-09-22T03:52:16.007 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosList.ListObjectsZeroInName 2024-09-22T03:52:16.381 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosList.ListObjectsZeroInName (374 ms) 2024-09-22T03:52:16.381 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosList.ListObjectsNS 2024-09-22T03:52:16.562 INFO:tasks.workunit.client.0.smithi003.stdout:myset foo1,foo2,foo3 2024-09-22T03:52:16.716 INFO:tasks.workunit.client.0.smithi003.stdout:foo1 2024-09-22T03:52:16.716 INFO:tasks.workunit.client.0.smithi003.stdout:foo2 2024-09-22T03:52:16.716 INFO:tasks.workunit.client.0.smithi003.stdout:foo3 2024-09-22T03:52:16.716 INFO:tasks.workunit.client.0.smithi003.stdout:myset foo1,foo4,foo5 2024-09-22T03:52:16.867 INFO:tasks.workunit.client.0.smithi003.stdout:foo4 2024-09-22T03:52:16.867 INFO:tasks.workunit.client.0.smithi003.stdout:foo5 2024-09-22T03:52:16.867 INFO:tasks.workunit.client.0.smithi003.stdout:foo1 2024-09-22T03:52:16.867 INFO:tasks.workunit.client.0.smithi003.stdout:myset foo6,foo7 2024-09-22T03:52:17.023 INFO:tasks.workunit.client.0.smithi003.stdout:foo7 2024-09-22T03:52:17.023 INFO:tasks.workunit.client.0.smithi003.stdout:foo6 2024-09-22T03:52:17.023 INFO:tasks.workunit.client.0.smithi003.stdout:myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-22T03:52:17.177 INFO:tasks.workunit.client.0.smithi003.stdout:ns1:foo4 2024-09-22T03:52:17.177 INFO:tasks.workunit.client.0.smithi003.stdout:ns1:foo5 2024-09-22T03:52:17.178 INFO:tasks.workunit.client.0.smithi003.stdout:ns2:foo7 2024-09-22T03:52:17.178 INFO:tasks.workunit.client.0.smithi003.stdout:ns2:foo6 2024-09-22T03:52:17.178 INFO:tasks.workunit.client.0.smithi003.stdout:ns1:foo1 2024-09-22T03:52:17.178 INFO:tasks.workunit.client.0.smithi003.stdout::foo1 2024-09-22T03:52:17.178 INFO:tasks.workunit.client.0.smithi003.stdout::foo2 2024-09-22T03:52:17.178 INFO:tasks.workunit.client.0.smithi003.stdout::foo3 2024-09-22T03:52:17.474 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosList.ListObjectsNS (1093 ms) 2024-09-22T03:52:17.474 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosList.ListObjectsStart 2024-09-22T03:52:18.020 INFO:tasks.workunit.client.0.smithi003.stdout:1 0 2024-09-22T03:52:18.021 INFO:tasks.workunit.client.0.smithi003.stdout:10 0 2024-09-22T03:52:18.021 INFO:tasks.workunit.client.0.smithi003.stdout:13 0 2024-09-22T03:52:18.021 INFO:tasks.workunit.client.0.smithi003.stdout:7 0 2024-09-22T03:52:18.021 INFO:tasks.workunit.client.0.smithi003.stdout:14 0 2024-09-22T03:52:18.021 INFO:tasks.workunit.client.0.smithi003.stdout:0 0 2024-09-22T03:52:18.021 INFO:tasks.workunit.client.0.smithi003.stdout:15 0 2024-09-22T03:52:18.021 INFO:tasks.workunit.client.0.smithi003.stdout:11 0 2024-09-22T03:52:18.021 INFO:tasks.workunit.client.0.smithi003.stdout:5 0 2024-09-22T03:52:18.021 INFO:tasks.workunit.client.0.smithi003.stdout:8 0 2024-09-22T03:52:18.021 INFO:tasks.workunit.client.0.smithi003.stdout:6 0 2024-09-22T03:52:18.021 INFO:tasks.workunit.client.0.smithi003.stdout:3 0 2024-09-22T03:52:18.021 INFO:tasks.workunit.client.0.smithi003.stdout:4 0 2024-09-22T03:52:18.021 INFO:tasks.workunit.client.0.smithi003.stdout:12 0 2024-09-22T03:52:18.022 INFO:tasks.workunit.client.0.smithi003.stdout:9 0 2024-09-22T03:52:18.022 INFO:tasks.workunit.client.0.smithi003.stdout:2 0 2024-09-22T03:52:18.173 INFO:tasks.workunit.client.0.smithi003.stdout:have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-22T03:52:18.624 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosList.ListObjectsStart (1151 ms) 2024-09-22T03:52:18.624 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosList.ListObjectsCursor 2024-09-22T03:52:18.975 INFO:tasks.workunit.client.0.smithi003.stdout:x cursor=MIN 2024-09-22T03:52:19.134 INFO:tasks.workunit.client.0.smithi003.stdout:> oid=1 cursor=76:02547ec2:::1:head 2024-09-22T03:52:19.134 INFO:tasks.workunit.client.0.smithi003.stdout:> oid=10 cursor=76:52ea6a34:::10:head 2024-09-22T03:52:19.135 INFO:tasks.workunit.client.0.smithi003.stdout:> oid=13 cursor=76:566253c9:::13:head 2024-09-22T03:52:19.135 INFO:tasks.workunit.client.0.smithi003.stdout:> oid=7 cursor=76:5c6b0b28:::7:head 2024-09-22T03:52:19.135 INFO:tasks.workunit.client.0.smithi003.stdout:> oid=14 cursor=76:62a1935d:::14:head 2024-09-22T03:52:19.135 INFO:tasks.workunit.client.0.smithi003.stdout:> oid=0 cursor=76:6cac518f:::0:head 2024-09-22T03:52:19.135 INFO:tasks.workunit.client.0.smithi003.stdout:> oid=15 cursor=76:863748b0:::15:head 2024-09-22T03:52:19.135 INFO:tasks.workunit.client.0.smithi003.stdout:> oid=11 cursor=76:89d3ae78:::11:head 2024-09-22T03:52:19.135 INFO:tasks.workunit.client.0.smithi003.stdout:> oid=5 cursor=76:b29083e3:::5:head 2024-09-22T03:52:19.135 INFO:tasks.workunit.client.0.smithi003.stdout:> oid=8 cursor=76:bd63b0f1:::8:head 2024-09-22T03:52:19.135 INFO:tasks.workunit.client.0.smithi003.stdout:> oid=6 cursor=76:c4fdafeb:::6:head 2024-09-22T03:52:19.135 INFO:tasks.workunit.client.0.smithi003.stdout:> oid=3 cursor=76:cfc208b3:::3:head 2024-09-22T03:52:19.135 INFO:tasks.workunit.client.0.smithi003.stdout:> oid=4 cursor=76:d83876eb:::4:head 2024-09-22T03:52:19.135 INFO:tasks.workunit.client.0.smithi003.stdout:> oid=12 cursor=76:de5d7c5f:::12:head 2024-09-22T03:52:19.135 INFO:tasks.workunit.client.0.smithi003.stdout:> oid=9 cursor=76:e960b815:::9:head 2024-09-22T03:52:19.135 INFO:tasks.workunit.client.0.smithi003.stdout:> oid=2 cursor=76:f905c69b:::2:head 2024-09-22T03:52:19.288 INFO:tasks.workunit.client.0.smithi003.stdout:FIRST> seek to MIN oid=1 2024-09-22T03:52:19.443 INFO:tasks.workunit.client.0.smithi003.stdout:: oid=1 cursor=76:02547ec2:::1:head 2024-09-22T03:52:19.443 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:02547ec2:::1:head 2024-09-22T03:52:19.597 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:02547ec2:::1:head -> 1 2024-09-22T03:52:19.597 INFO:tasks.workunit.client.0.smithi003.stdout:: oid=10 cursor=76:52ea6a34:::10:head 2024-09-22T03:52:19.597 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:52ea6a34:::10:head 2024-09-22T03:52:19.702 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:52ea6a34:::10:head -> 10 2024-09-22T03:52:19.702 INFO:tasks.workunit.client.0.smithi003.stdout:: oid=13 cursor=76:566253c9:::13:head 2024-09-22T03:52:19.702 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:566253c9:::13:head 2024-09-22T03:52:19.806 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:566253c9:::13:head -> 13 2024-09-22T03:52:19.806 INFO:tasks.workunit.client.0.smithi003.stdout:: oid=7 cursor=76:5c6b0b28:::7:head 2024-09-22T03:52:19.806 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:5c6b0b28:::7:head 2024-09-22T03:52:19.906 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:5c6b0b28:::7:head -> 7 2024-09-22T03:52:19.907 INFO:tasks.workunit.client.0.smithi003.stdout:: oid=14 cursor=76:62a1935d:::14:head 2024-09-22T03:52:19.907 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:62a1935d:::14:head 2024-09-22T03:52:20.002 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:62a1935d:::14:head -> 14 2024-09-22T03:52:20.085 INFO:tasks.workunit.client.0.smithi003.stdout:: oid=0 cursor=76:6cac518f:::0:head 2024-09-22T03:52:20.085 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:6cac518f:::0:head 2024-09-22T03:52:20.093 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:6cac518f:::0:head -> 0 2024-09-22T03:52:20.093 INFO:tasks.workunit.client.0.smithi003.stdout:: oid=15 cursor=76:863748b0:::15:head 2024-09-22T03:52:20.093 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:863748b0:::15:head 2024-09-22T03:52:20.169 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:863748b0:::15:head -> 15 2024-09-22T03:52:20.169 INFO:tasks.workunit.client.0.smithi003.stdout:: oid=11 cursor=76:89d3ae78:::11:head 2024-09-22T03:52:20.169 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:89d3ae78:::11:head 2024-09-22T03:52:20.239 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:89d3ae78:::11:head -> 11 2024-09-22T03:52:20.240 INFO:tasks.workunit.client.0.smithi003.stdout:: oid=5 cursor=76:b29083e3:::5:head 2024-09-22T03:52:20.240 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:b29083e3:::5:head 2024-09-22T03:52:20.288 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:b29083e3:::5:head -> 5 2024-09-22T03:52:20.288 INFO:tasks.workunit.client.0.smithi003.stdout:: oid=8 cursor=76:bd63b0f1:::8:head 2024-09-22T03:52:20.288 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:bd63b0f1:::8:head 2024-09-22T03:52:20.332 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:bd63b0f1:::8:head -> 8 2024-09-22T03:52:20.332 INFO:tasks.workunit.client.0.smithi003.stdout:: oid=6 cursor=76:c4fdafeb:::6:head 2024-09-22T03:52:20.332 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:c4fdafeb:::6:head 2024-09-22T03:52:20.371 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:c4fdafeb:::6:head -> 6 2024-09-22T03:52:20.371 INFO:tasks.workunit.client.0.smithi003.stdout:: oid=3 cursor=76:cfc208b3:::3:head 2024-09-22T03:52:20.372 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:cfc208b3:::3:head 2024-09-22T03:52:20.405 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:cfc208b3:::3:head -> 3 2024-09-22T03:52:20.406 INFO:tasks.workunit.client.0.smithi003.stdout:: oid=4 cursor=76:d83876eb:::4:head 2024-09-22T03:52:20.406 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:d83876eb:::4:head 2024-09-22T03:52:20.430 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:d83876eb:::4:head -> 4 2024-09-22T03:52:20.430 INFO:tasks.workunit.client.0.smithi003.stdout:: oid=12 cursor=76:de5d7c5f:::12:head 2024-09-22T03:52:20.430 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:de5d7c5f:::12:head 2024-09-22T03:52:20.455 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:de5d7c5f:::12:head -> 12 2024-09-22T03:52:20.455 INFO:tasks.workunit.client.0.smithi003.stdout:: oid=9 cursor=76:e960b815:::9:head 2024-09-22T03:52:20.456 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:e960b815:::9:head 2024-09-22T03:52:20.470 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:e960b815:::9:head -> 9 2024-09-22T03:52:20.470 INFO:tasks.workunit.client.0.smithi003.stdout:: oid=2 cursor=76:f905c69b:::2:head 2024-09-22T03:52:20.470 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:f905c69b:::2:head 2024-09-22T03:52:20.474 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:f905c69b:::2:head -> 2 2024-09-22T03:52:20.475 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:52ea6a34:::10:head 2024-09-22T03:52:20.475 INFO:tasks.workunit.client.0.smithi003.stdout:: cursor()=76:52ea6a34:::10:head expected=76:52ea6a34:::10:head 2024-09-22T03:52:20.583 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:52ea6a34:::10:head -> 10 2024-09-22T03:52:20.584 INFO:tasks.workunit.client.0.smithi003.stdout:: entry=10 expected=10 2024-09-22T03:52:20.584 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:6cac518f:::0:head 2024-09-22T03:52:20.584 INFO:tasks.workunit.client.0.smithi003.stdout:: cursor()=76:6cac518f:::0:head expected=76:6cac518f:::0:head 2024-09-22T03:52:20.675 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:6cac518f:::0:head -> 0 2024-09-22T03:52:20.676 INFO:tasks.workunit.client.0.smithi003.stdout:: entry=0 expected=0 2024-09-22T03:52:20.676 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:62a1935d:::14:head 2024-09-22T03:52:20.676 INFO:tasks.workunit.client.0.smithi003.stdout:: cursor()=76:62a1935d:::14:head expected=76:62a1935d:::14:head 2024-09-22T03:52:20.772 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:62a1935d:::14:head -> 14 2024-09-22T03:52:20.772 INFO:tasks.workunit.client.0.smithi003.stdout:: entry=14 expected=14 2024-09-22T03:52:20.772 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:de5d7c5f:::12:head 2024-09-22T03:52:20.772 INFO:tasks.workunit.client.0.smithi003.stdout:: cursor()=76:de5d7c5f:::12:head expected=76:de5d7c5f:::12:head 2024-09-22T03:52:20.796 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:de5d7c5f:::12:head -> 12 2024-09-22T03:52:20.796 INFO:tasks.workunit.client.0.smithi003.stdout:: entry=12 expected=12 2024-09-22T03:52:20.796 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:d83876eb:::4:head 2024-09-22T03:52:20.797 INFO:tasks.workunit.client.0.smithi003.stdout:: cursor()=76:d83876eb:::4:head expected=76:d83876eb:::4:head 2024-09-22T03:52:20.821 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:d83876eb:::4:head -> 4 2024-09-22T03:52:20.821 INFO:tasks.workunit.client.0.smithi003.stdout:: entry=4 expected=4 2024-09-22T03:52:20.821 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:f905c69b:::2:head 2024-09-22T03:52:20.821 INFO:tasks.workunit.client.0.smithi003.stdout:: cursor()=76:f905c69b:::2:head expected=76:f905c69b:::2:head 2024-09-22T03:52:20.825 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:f905c69b:::2:head -> 2 2024-09-22T03:52:20.825 INFO:tasks.workunit.client.0.smithi003.stdout:: entry=2 expected=2 2024-09-22T03:52:20.825 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:e960b815:::9:head 2024-09-22T03:52:20.825 INFO:tasks.workunit.client.0.smithi003.stdout:: cursor()=76:e960b815:::9:head expected=76:e960b815:::9:head 2024-09-22T03:52:20.839 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:e960b815:::9:head -> 9 2024-09-22T03:52:20.840 INFO:tasks.workunit.client.0.smithi003.stdout:: entry=9 expected=9 2024-09-22T03:52:20.840 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:02547ec2:::1:head 2024-09-22T03:52:20.840 INFO:tasks.workunit.client.0.smithi003.stdout:: cursor()=76:02547ec2:::1:head expected=76:02547ec2:::1:head 2024-09-22T03:52:21.003 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:02547ec2:::1:head -> 1 2024-09-22T03:52:21.003 INFO:tasks.workunit.client.0.smithi003.stdout:: entry=1 expected=1 2024-09-22T03:52:21.003 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:cfc208b3:::3:head 2024-09-22T03:52:21.003 INFO:tasks.workunit.client.0.smithi003.stdout:: cursor()=76:cfc208b3:::3:head expected=76:cfc208b3:::3:head 2024-09-22T03:52:21.037 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:cfc208b3:::3:head -> 3 2024-09-22T03:52:21.037 INFO:tasks.workunit.client.0.smithi003.stdout:: entry=3 expected=3 2024-09-22T03:52:21.037 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:c4fdafeb:::6:head 2024-09-22T03:52:21.038 INFO:tasks.workunit.client.0.smithi003.stdout:: cursor()=76:c4fdafeb:::6:head expected=76:c4fdafeb:::6:head 2024-09-22T03:52:21.076 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:c4fdafeb:::6:head -> 6 2024-09-22T03:52:21.076 INFO:tasks.workunit.client.0.smithi003.stdout:: entry=6 expected=6 2024-09-22T03:52:21.076 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:bd63b0f1:::8:head 2024-09-22T03:52:21.076 INFO:tasks.workunit.client.0.smithi003.stdout:: cursor()=76:bd63b0f1:::8:head expected=76:bd63b0f1:::8:head 2024-09-22T03:52:21.120 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:bd63b0f1:::8:head -> 8 2024-09-22T03:52:21.120 INFO:tasks.workunit.client.0.smithi003.stdout:: entry=8 expected=8 2024-09-22T03:52:21.120 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:b29083e3:::5:head 2024-09-22T03:52:21.120 INFO:tasks.workunit.client.0.smithi003.stdout:: cursor()=76:b29083e3:::5:head expected=76:b29083e3:::5:head 2024-09-22T03:52:21.167 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:b29083e3:::5:head -> 5 2024-09-22T03:52:21.167 INFO:tasks.workunit.client.0.smithi003.stdout:: entry=5 expected=5 2024-09-22T03:52:21.167 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:89d3ae78:::11:head 2024-09-22T03:52:21.168 INFO:tasks.workunit.client.0.smithi003.stdout:: cursor()=76:89d3ae78:::11:head expected=76:89d3ae78:::11:head 2024-09-22T03:52:21.238 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:89d3ae78:::11:head -> 11 2024-09-22T03:52:21.238 INFO:tasks.workunit.client.0.smithi003.stdout:: entry=11 expected=11 2024-09-22T03:52:21.238 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:863748b0:::15:head 2024-09-22T03:52:21.238 INFO:tasks.workunit.client.0.smithi003.stdout:: cursor()=76:863748b0:::15:head expected=76:863748b0:::15:head 2024-09-22T03:52:21.313 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:863748b0:::15:head -> 15 2024-09-22T03:52:21.313 INFO:tasks.workunit.client.0.smithi003.stdout:: entry=15 expected=15 2024-09-22T03:52:21.313 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:5c6b0b28:::7:head 2024-09-22T03:52:21.313 INFO:tasks.workunit.client.0.smithi003.stdout:: cursor()=76:5c6b0b28:::7:head expected=76:5c6b0b28:::7:head 2024-09-22T03:52:21.423 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:5c6b0b28:::7:head -> 7 2024-09-22T03:52:21.423 INFO:tasks.workunit.client.0.smithi003.stdout:: entry=7 expected=7 2024-09-22T03:52:21.423 INFO:tasks.workunit.client.0.smithi003.stdout:: seek to 76:566253c9:::13:head 2024-09-22T03:52:21.424 INFO:tasks.workunit.client.0.smithi003.stdout:: cursor()=76:566253c9:::13:head expected=76:566253c9:::13:head 2024-09-22T03:52:21.536 INFO:tasks.workunit.client.0.smithi003.stdout:> 76:566253c9:::13:head -> 13 2024-09-22T03:52:21.536 INFO:tasks.workunit.client.0.smithi003.stdout:: entry=13 expected=13 2024-09-22T03:52:22.000 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosList.ListObjectsCursor (3376 ms) 2024-09-22T03:52:22.000 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosList.EnumerateObjects 2024-09-22T03:52:22.971 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosList.EnumerateObjects (971 ms) 2024-09-22T03:52:22.971 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosList.EnumerateObjectsSplit 2024-09-22T03:52:23.316 INFO:tasks.workunit.client.0.smithi003.stdout:split 0/5 -> MIN 76:33333333::::head 2024-09-22T03:52:23.348 INFO:tasks.workunit.client.0.smithi003.stdout:split 1/5 -> 76:33333333::::head 76:66666666::::head 2024-09-22T03:52:23.385 INFO:tasks.workunit.client.0.smithi003.stdout:split 2/5 -> 76:66666666::::head 76:99999999::::head 2024-09-22T03:52:23.434 INFO:tasks.workunit.client.0.smithi003.stdout:split 3/5 -> 76:99999999::::head 76:cccccccc::::head 2024-09-22T03:52:23.474 INFO:tasks.workunit.client.0.smithi003.stdout:split 4/5 -> 76:cccccccc::::head MAX 2024-09-22T03:52:23.987 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosList.EnumerateObjectsSplit (1016 ms) 2024-09-22T03:52:24.223 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 7 tests from LibRadosList (8745 ms total) 2024-09-22T03:52:24.223 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:52:24.223 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 3 tests from LibRadosListEC 2024-09-22T03:52:24.223 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:70: Skipped 2024-09-22T03:52:24.223 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:24.223 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosListEC.ListObjects 2024-09-22T03:52:24.223 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:84: Skipped 2024-09-22T03:52:24.223 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:24.223 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:96: Skipped 2024-09-22T03:52:24.224 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:24.224 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosListEC.ListObjects (0 ms) 2024-09-22T03:52:24.224 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosListEC.ListObjectsNS 2024-09-22T03:52:24.224 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:84: Skipped 2024-09-22T03:52:24.224 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:24.224 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:96: Skipped 2024-09-22T03:52:24.224 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:24.224 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsNS (0 ms) 2024-09-22T03:52:24.224 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosListEC.ListObjectsStart 2024-09-22T03:52:24.224 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:84: Skipped 2024-09-22T03:52:24.224 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:24.224 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:96: Skipped 2024-09-22T03:52:24.224 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:24.224 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsStart (0 ms) 2024-09-22T03:52:24.224 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:78: Skipped 2024-09-22T03:52:24.225 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:24.225 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 3 tests from LibRadosListEC (0 ms total) 2024-09-22T03:52:24.225 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:52:24.225 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibRadosListNP 2024-09-22T03:52:24.225 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosListNP.ListObjectsError 2024-09-22T03:52:27.238 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosListNP.ListObjectsError (3016 ms) 2024-09-22T03:52:27.238 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibRadosListNP (3016 ms total) 2024-09-22T03:52:27.238 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:52:27.239 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:52:27.239 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 11 tests from 3 test suites ran. (13409 ms total) 2024-09-22T03:52:27.239 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 8 tests. 2024-09-22T03:52:27.239 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] 3 tests, listed below: 2024-09-22T03:52:27.239 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosListEC.ListObjects 2024-09-22T03:52:27.239 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsNS 2024-09-22T03:52:27.239 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosListEC.ListObjectsStart 2024-09-22T03:52:27.242 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:52:27.243 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:52:27.243 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_lock 2024-09-22T03:52:27.265 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:52:27.266 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 16 tests from 2 test suites. 2024-09-22T03:52:27.266 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:52:27.266 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 8 tests from LibRadosLock 2024-09-22T03:52:29.252 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLock.LockExclusive 2024-09-22T03:52:29.638 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosLock.LockExclusive (386 ms) 2024-09-22T03:52:29.638 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLock.LockShared 2024-09-22T03:52:29.728 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosLock.LockShared (90 ms) 2024-09-22T03:52:29.729 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLock.LockExclusiveDur 2024-09-22T03:52:30.790 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosLock.LockExclusiveDur (1062 ms) 2024-09-22T03:52:30.791 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLock.LockSharedDur 2024-09-22T03:52:31.854 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosLock.LockSharedDur (1064 ms) 2024-09-22T03:52:31.854 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLock.LockMayRenew 2024-09-22T03:52:31.919 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosLock.LockMayRenew (64 ms) 2024-09-22T03:52:31.919 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLock.Unlock 2024-09-22T03:52:31.987 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosLock.Unlock (69 ms) 2024-09-22T03:52:31.987 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLock.ListLockers 2024-09-22T03:52:32.069 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosLock.ListLockers (82 ms) 2024-09-22T03:52:32.069 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLock.BreakLock 2024-09-22T03:52:32.121 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosLock.BreakLock (52 ms) 2024-09-22T03:52:32.263 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 8 tests from LibRadosLock (2869 ms total) 2024-09-22T03:52:32.264 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:52:32.264 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 8 tests from LibRadosLockEC 2024-09-22T03:52:32.264 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:168: Skipped 2024-09-22T03:52:32.264 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockEC.LockExclusive 2024-09-22T03:52:32.264 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:32.264 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.264 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:32.264 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusive (0 ms) 2024-09-22T03:52:32.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockEC.LockShared 2024-09-22T03:52:32.264 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:32.264 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.265 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:32.265 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.265 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockEC.LockShared (0 ms) 2024-09-22T03:52:32.265 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockEC.LockExclusiveDur 2024-09-22T03:52:32.265 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:32.265 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.265 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:32.265 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.265 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusiveDur (0 ms) 2024-09-22T03:52:32.265 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockEC.LockSharedDur 2024-09-22T03:52:32.265 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:32.265 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.265 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:32.266 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.266 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockEC.LockSharedDur (0 ms) 2024-09-22T03:52:32.266 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockEC.LockMayRenew 2024-09-22T03:52:32.266 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:32.266 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.266 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:32.266 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.266 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockEC.LockMayRenew (0 ms) 2024-09-22T03:52:32.266 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockEC.Unlock 2024-09-22T03:52:32.266 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:32.266 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.266 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:32.266 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.266 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockEC.Unlock (0 ms) 2024-09-22T03:52:32.267 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockEC.ListLockers 2024-09-22T03:52:32.267 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:32.267 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.267 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:32.267 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.267 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockEC.ListLockers (0 ms) 2024-09-22T03:52:32.267 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockEC.BreakLock 2024-09-22T03:52:32.267 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:52:32.267 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.267 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:52:32.267 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.267 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockEC.BreakLock (0 ms) 2024-09-22T03:52:32.267 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:176: Skipped 2024-09-22T03:52:32.267 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:32.267 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 8 tests from LibRadosLockEC (0 ms total) 2024-09-22T03:52:32.268 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:52:32.268 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:52:32.268 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 16 tests from 2 test suites ran. (4997 ms total) 2024-09-22T03:52:32.268 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 8 tests. 2024-09-22T03:52:32.268 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] 8 tests, listed below: 2024-09-22T03:52:32.268 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusive 2024-09-22T03:52:32.268 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockEC.LockShared 2024-09-22T03:52:32.268 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockEC.LockExclusiveDur 2024-09-22T03:52:32.268 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockEC.LockSharedDur 2024-09-22T03:52:32.268 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockEC.LockMayRenew 2024-09-22T03:52:32.268 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockEC.Unlock 2024-09-22T03:52:32.268 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockEC.ListLockers 2024-09-22T03:52:32.268 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockEC.BreakLock 2024-09-22T03:52:32.269 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:52:32.269 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:52:32.269 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_lock_pp 2024-09-22T03:52:32.287 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:52:32.287 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 16 tests from 2 test suites. 2024-09-22T03:52:32.287 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:52:32.287 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 8 tests from LibRadosLockPP 2024-09-22T03:52:32.287 INFO:tasks.workunit.client.0.smithi003.stdout:seed 31036 2024-09-22T03:52:34.278 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockPP.LockExclusivePP 2024-09-22T03:52:34.643 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosLockPP.LockExclusivePP (367 ms) 2024-09-22T03:52:34.643 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockPP.LockSharedPP 2024-09-22T03:52:34.695 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosLockPP.LockSharedPP (52 ms) 2024-09-22T03:52:34.695 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-09-22T03:52:35.757 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosLockPP.LockExclusiveDurPP (1062 ms) 2024-09-22T03:52:35.757 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockPP.LockSharedDurPP 2024-09-22T03:52:36.826 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosLockPP.LockSharedDurPP (1068 ms) 2024-09-22T03:52:36.826 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockPP.LockMayRenewPP 2024-09-22T03:52:36.894 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosLockPP.LockMayRenewPP (69 ms) 2024-09-22T03:52:36.895 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockPP.UnlockPP 2024-09-22T03:52:36.963 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosLockPP.UnlockPP (68 ms) 2024-09-22T03:52:36.963 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockPP.ListLockersPP 2024-09-22T03:52:37.046 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosLockPP.ListLockersPP (84 ms) 2024-09-22T03:52:37.046 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockPP.BreakLockPP 2024-09-22T03:52:37.098 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosLockPP.BreakLockPP (52 ms) 2024-09-22T03:52:37.287 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 8 tests from LibRadosLockPP (2822 ms total) 2024-09-22T03:52:37.288 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:52:37.288 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 8 tests from LibRadosLockECPP 2024-09-22T03:52:37.288 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:375: Skipped 2024-09-22T03:52:37.288 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.288 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockECPP.LockExclusivePP 2024-09-22T03:52:37.288 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:37.288 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.288 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:37.288 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.288 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusivePP (0 ms) 2024-09-22T03:52:37.288 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockECPP.LockSharedPP 2024-09-22T03:52:37.288 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:37.288 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.288 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:37.289 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.289 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedPP (0 ms) 2024-09-22T03:52:37.289 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-09-22T03:52:37.289 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:37.289 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.289 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:37.289 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.289 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusiveDurPP (0 ms) 2024-09-22T03:52:37.289 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-09-22T03:52:37.289 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:37.289 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.289 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:37.289 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.289 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedDurPP (0 ms) 2024-09-22T03:52:37.290 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-09-22T03:52:37.290 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:37.290 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.290 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:37.290 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.290 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockECPP.LockMayRenewPP (0 ms) 2024-09-22T03:52:37.290 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockECPP.UnlockPP 2024-09-22T03:52:37.290 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:37.290 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.290 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:37.290 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.290 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockECPP.UnlockPP (0 ms) 2024-09-22T03:52:37.290 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockECPP.ListLockersPP 2024-09-22T03:52:37.291 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:37.291 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.291 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:37.291 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.291 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockECPP.ListLockersPP (0 ms) 2024-09-22T03:52:37.291 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosLockECPP.BreakLockPP 2024-09-22T03:52:37.291 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:52:37.291 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.291 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:52:37.291 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.291 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockECPP.BreakLockPP (0 ms) 2024-09-22T03:52:37.291 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:383: Skipped 2024-09-22T03:52:37.291 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:52:37.291 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 8 tests from LibRadosLockECPP (0 ms total) 2024-09-22T03:52:37.291 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:52:37.291 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:52:37.292 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 16 tests from 2 test suites ran. (4999 ms total) 2024-09-22T03:52:37.292 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 8 tests. 2024-09-22T03:52:37.292 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] 8 tests, listed below: 2024-09-22T03:52:37.292 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusivePP 2024-09-22T03:52:37.292 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedPP 2024-09-22T03:52:37.292 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockECPP.LockExclusiveDurPP 2024-09-22T03:52:37.292 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockECPP.LockSharedDurPP 2024-09-22T03:52:37.292 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockECPP.LockMayRenewPP 2024-09-22T03:52:37.292 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockECPP.UnlockPP 2024-09-22T03:52:37.292 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockECPP.ListLockersPP 2024-09-22T03:52:37.292 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosLockECPP.BreakLockPP 2024-09-22T03:52:37.293 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:52:37.293 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:52:37.293 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_misc 2024-09-22T03:52:37.326 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:37.324+0000 7f574b4906c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:52:37.326 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:37.324+0000 7f574b4906c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:52:37.326 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:37.325+0000 7f574b4906c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:52:37.326 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 12 tests from 4 test suites. 2024-09-22T03:52:37.327 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:52:37.327 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibRadosMiscVersion 2024-09-22T03:52:37.327 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscVersion.Version 2024-09-22T03:52:37.327 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscVersion.Version (0 ms) 2024-09-22T03:52:37.327 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-22T03:52:37.327 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:52:37.327 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 2 tests from LibRadosMiscConnectFailure 2024-09-22T03:52:37.327 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-09-22T03:52:37.393 INFO:tasks.workunit.client.0.smithi003.stderr:unable to get monitor info from DNS SRV with service name: ceph-mon 2024-09-22T03:52:37.393 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:37.392+0000 7f574b4906c0 -1 failed for service _ceph-mon._tcp 2024-09-22T03:52:37.393 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:52:37.392+0000 7f574b4906c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-09-22T03:52:37.393 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscConnectFailure.ConnectFailure (67 ms) 2024-09-22T03:52:37.394 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-09-22T03:52:42.402 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscConnectFailure.ConnectTimeout (5009 ms) 2024-09-22T03:52:42.402 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 2 tests from LibRadosMiscConnectFailure (5076 ms total) 2024-09-22T03:52:42.402 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:52:42.402 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibRadosMiscPool 2024-09-22T03:52:42.402 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPool.PoolCreationRace 2024-09-22T03:52:44.225 INFO:tasks.workunit.client.0.smithi003.stdout:started 0x7f5714005d20 2024-09-22T03:52:45.226 INFO:tasks.workunit.client.0.smithi003.stdout:started 0x7f5734005d20 2024-09-22T03:52:45.226 INFO:tasks.workunit.client.0.smithi003.stdout: started 2 aios 2024-09-22T03:52:45.226 INFO:tasks.workunit.client.0.smithi003.stdout:waiting 0x7f5714005d20 2024-09-22T03:52:45.226 INFO:tasks.workunit.client.0.smithi003.stdout:waiting 0x7f5734005d20 2024-09-22T03:52:45.230 INFO:tasks.workunit.client.0.smithi003.stdout:done. 2024-09-22T03:52:47.246 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPool.PoolCreationRace (4844 ms) 2024-09-22T03:52:47.246 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibRadosMiscPool (4844 ms total) 2024-09-22T03:52:47.247 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:52:47.247 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 8 tests from LibRadosMisc 2024-09-22T03:52:49.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMisc.ClusterFSID 2024-09-22T03:52:49.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-09-22T03:52:49.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMisc.Exec 2024-09-22T03:52:49.639 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMisc.Exec (379 ms) 2024-09-22T03:52:49.639 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMisc.WriteSame 2024-09-22T03:52:49.763 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMisc.WriteSame (123 ms) 2024-09-22T03:52:49.763 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMisc.CmpExt 2024-09-22T03:52:49.805 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMisc.CmpExt (42 ms) 2024-09-22T03:52:49.805 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMisc.Applications 2024-09-22T03:52:54.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMisc.Applications (4425 ms) 2024-09-22T03:52:54.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMisc.MinCompatOSD 2024-09-22T03:52:54.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-09-22T03:52:54.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMisc.MinCompatClient 2024-09-22T03:52:54.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-09-22T03:52:54.229 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMisc.ShutdownRace 2024-09-22T03:53:22.910 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMisc.ShutdownRace (28680 ms) 2024-09-22T03:53:23.249 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 8 tests from LibRadosMisc (33649 ms total) 2024-09-22T03:53:23.249 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:53:23.249 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:53:23.249 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 12 tests from 4 test suites ran. (45923 ms total) 2024-09-22T03:53:23.249 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 12 tests. 2024-09-22T03:53:23.260 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:53:23.260 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:53:23.260 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_misc_pp 2024-09-22T03:53:23.307 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:53:23.305+0000 7f69b0af4840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:53:23.307 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:53:23.306+0000 7f69b0af4840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:53:23.307 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:53:23.306+0000 7f69b0af4840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:53:23.308 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 31 tests from 7 test suites. 2024-09-22T03:53:23.308 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:53:23.308 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibRadosMiscVersion 2024-09-22T03:53:23.308 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscVersion.VersionPP 2024-09-22T03:53:23.309 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-09-22T03:53:23.309 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-22T03:53:23.309 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:53:23.309 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 22 tests from LibRadosMiscPP 2024-09-22T03:53:23.309 INFO:tasks.workunit.client.0.smithi003.stdout:seed 92701 2024-09-22T03:53:25.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-09-22T03:53:25.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.WaitOSDMapPP (1 ms) 2024-09-22T03:53:25.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.LongNamePP 2024-09-22T03:53:25.827 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.LongNamePP (567 ms) 2024-09-22T03:53:25.828 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.LongLocatorPP 2024-09-22T03:53:25.828 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/misc_cxx.cc:53: Skipped 2024-09-22T03:53:25.828 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:53:25.828 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosMiscPP.LongLocatorPP (0 ms) 2024-09-22T03:53:25.828 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.LongNSpacePP 2024-09-22T03:53:25.927 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.LongNSpacePP (99 ms) 2024-09-22T03:53:25.927 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-09-22T03:53:25.999 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.LongAttrNamePP (73 ms) 2024-09-22T03:53:25.999 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.ExecPP 2024-09-22T03:53:26.028 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.ExecPP (29 ms) 2024-09-22T03:53:26.028 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.BadFlagsPP 2024-09-22T03:53:26.059 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.BadFlagsPP (31 ms) 2024-09-22T03:53:26.059 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.Operate1PP 2024-09-22T03:53:26.105 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.Operate1PP (46 ms) 2024-09-22T03:53:26.105 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.Operate2PP 2024-09-22T03:53:26.137 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.Operate2PP (31 ms) 2024-09-22T03:53:26.137 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.BigObjectPP 2024-09-22T03:53:26.213 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.BigObjectPP (76 ms) 2024-09-22T03:53:26.213 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.AioOperatePP 2024-09-22T03:53:26.241 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.AioOperatePP (29 ms) 2024-09-22T03:53:26.241 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.AssertExistsPP 2024-09-22T03:53:26.284 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.AssertExistsPP (42 ms) 2024-09-22T03:53:26.284 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.AssertVersionPP 2024-09-22T03:53:26.417 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.AssertVersionPP (134 ms) 2024-09-22T03:53:26.417 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.BigAttrPP 2024-09-22T03:53:26.438 INFO:tasks.workunit.client.0.smithi003.stdout:osd_max_attr_size = 0 2024-09-22T03:53:26.438 INFO:tasks.workunit.client.0.smithi003.stdout:osd_max_attr_size == 0; skipping test 2024-09-22T03:53:53.730 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.BigAttrPP (27312 ms) 2024-09-22T03:53:53.730 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.CopyPP 2024-09-22T03:53:53.730 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/misc_cxx.cc:387: Skipped 2024-09-22T03:53:53.730 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:53:53.730 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosMiscPP.CopyPP (0 ms) 2024-09-22T03:53:53.730 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.CopyScrubPP 2024-09-22T03:53:53.730 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/misc_cxx.cc:550: Skipped 2024-09-22T03:53:53.730 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:53:53.730 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosMiscPP.CopyScrubPP (0 ms) 2024-09-22T03:53:53.730 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.WriteSamePP 2024-09-22T03:53:53.803 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.WriteSamePP (73 ms) 2024-09-22T03:53:53.803 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.CmpExtPP 2024-09-22T03:53:53.841 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.CmpExtPP (38 ms) 2024-09-22T03:53:53.841 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.Applications 2024-09-22T03:53:58.267 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.Applications (4426 ms) 2024-09-22T03:53:58.267 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.MinCompatOSD 2024-09-22T03:53:58.267 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-09-22T03:53:58.267 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.MinCompatClient 2024-09-22T03:53:58.267 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-09-22T03:53:58.267 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscPP.Conf 2024-09-22T03:53:58.267 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosMiscPP.Conf (0 ms) 2024-09-22T03:53:59.267 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 22 tests from LibRadosMiscPP (33007 ms total) 2024-09-22T03:53:59.267 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:53:59.267 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibRadosTwoPoolsECPP 2024-09-22T03:53:59.267 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/misc_cxx.cc:477: Skipped 2024-09-22T03:53:59.267 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:53:59.267 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-09-22T03:53:59.267 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/misc_cxx.cc:499: Skipped 2024-09-22T03:53:59.267 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:53:59.267 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/misc_cxx.cc:505: Skipped 2024-09-22T03:53:59.268 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:53:59.268 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CopyFrom (0 ms) 2024-09-22T03:53:59.268 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/misc_cxx.cc:492: Skipped 2024-09-22T03:53:59.268 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:53:59.268 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibRadosTwoPoolsECPP (0 ms total) 2024-09-22T03:53:59.268 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:53:59.268 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-09-22T03:54:01.278 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosChecksum/0.Subset 2024-09-22T03:54:01.659 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosChecksum/0.Subset (380 ms) 2024-09-22T03:54:01.659 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosChecksum/0.Chunked 2024-09-22T03:54:01.722 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosChecksum/0.Chunked (63 ms) 2024-09-22T03:54:02.282 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 2 tests from LibRadosChecksum/0 (443 ms total) 2024-09-22T03:54:02.282 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:54:02.282 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-09-22T03:54:04.245 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosChecksum/1.Subset 2024-09-22T03:54:04.586 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosChecksum/1.Subset (340 ms) 2024-09-22T03:54:04.586 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosChecksum/1.Chunked 2024-09-22T03:54:04.677 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosChecksum/1.Chunked (92 ms) 2024-09-22T03:54:05.290 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 2 tests from LibRadosChecksum/1 (432 ms total) 2024-09-22T03:54:05.291 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:54:05.291 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-09-22T03:54:07.301 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosChecksum/2.Subset 2024-09-22T03:54:07.652 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosChecksum/2.Subset (351 ms) 2024-09-22T03:54:07.652 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosChecksum/2.Chunked 2024-09-22T03:54:07.691 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosChecksum/2.Chunked (39 ms) 2024-09-22T03:54:08.306 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 2 tests from LibRadosChecksum/2 (390 ms total) 2024-09-22T03:54:08.306 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:54:08.306 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibRadosMiscECPP 2024-09-22T03:54:08.306 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:375: Skipped 2024-09-22T03:54:08.306 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:08.306 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-09-22T03:54:08.307 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:54:08.307 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:08.307 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:54:08.307 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:08.307 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosMiscECPP.CompareExtentRange (0 ms) 2024-09-22T03:54:08.307 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:383: Skipped 2024-09-22T03:54:08.307 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:08.307 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibRadosMiscECPP (0 ms total) 2024-09-22T03:54:08.307 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:54:08.307 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:54:08.307 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 31 tests from 7 test suites ran. (44998 ms total) 2024-09-22T03:54:08.307 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 26 tests. 2024-09-22T03:54:08.307 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] 5 tests, listed below: 2024-09-22T03:54:08.307 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosMiscPP.LongLocatorPP 2024-09-22T03:54:08.308 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosMiscPP.CopyPP 2024-09-22T03:54:08.308 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosMiscPP.CopyScrubPP 2024-09-22T03:54:08.308 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CopyFrom 2024-09-22T03:54:08.308 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosMiscECPP.CompareExtentRange 2024-09-22T03:54:08.308 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:54:08.309 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:54:08.309 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_tier_pp 2024-09-22T03:54:08.349 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:54:08.347+0000 7fe7540dd9c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:54:08.349 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:54:08.347+0000 7fe7540dd9c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:54:08.349 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:54:08.348+0000 7fe7540dd9c0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:54:08.349 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 77 tests from 4 test suites. 2024-09-22T03:54:08.349 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:54:08.349 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 3 tests from LibRadosTierPP 2024-09-22T03:54:08.350 INFO:tasks.workunit.client.0.smithi003.stdout:seed 92775 2024-09-22T03:54:10.316 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTierPP.Dirty 2024-09-22T03:54:10.316 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:287: Skipped 2024-09-22T03:54:10.316 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:10.316 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTierPP.Dirty (0 ms) 2024-09-22T03:54:10.316 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTierPP.FlushWriteRaces 2024-09-22T03:54:10.316 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:2182: Skipped 2024-09-22T03:54:10.317 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:10.317 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTierPP.FlushWriteRaces (0 ms) 2024-09-22T03:54:10.317 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTierPP.HitSetNone 2024-09-22T03:54:10.317 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:2561: Skipped 2024-09-22T03:54:10.317 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:10.317 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTierPP.HitSetNone (0 ms) 2024-09-22T03:54:11.318 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 3 tests from LibRadosTierPP (0 ms total) 2024-09-22T03:54:11.318 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:54:11.318 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 48 tests from LibRadosTwoPoolsPP 2024-09-22T03:54:13.253 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.Overlay 2024-09-22T03:54:13.254 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.254 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.254 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.254 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.254 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Overlay (0 ms) 2024-09-22T03:54:13.254 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.Promote 2024-09-22T03:54:13.254 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.254 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.254 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.254 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.254 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Promote (0 ms) 2024-09-22T03:54:13.254 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-09-22T03:54:13.254 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.254 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.255 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.255 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnap (0 ms) 2024-09-22T03:54:13.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-09-22T03:54:13.255 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.255 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.255 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.255 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapScrub (0 ms) 2024-09-22T03:54:13.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-09-22T03:54:13.255 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.255 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.255 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.255 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (0 ms) 2024-09-22T03:54:13.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-09-22T03:54:13.256 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.256 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.256 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.256 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Whiteout (0 ms) 2024-09-22T03:54:13.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-09-22T03:54:13.256 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.256 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.256 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.256 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (0 ms) 2024-09-22T03:54:13.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.Evict 2024-09-22T03:54:13.257 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.257 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.257 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.257 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.257 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Evict (0 ms) 2024-09-22T03:54:13.257 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-09-22T03:54:13.257 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.257 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.257 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.257 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.257 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap (0 ms) 2024-09-22T03:54:13.257 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-09-22T03:54:13.257 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.257 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.258 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.258 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.258 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap2 (0 ms) 2024-09-22T03:54:13.258 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-09-22T03:54:13.258 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.258 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.258 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.258 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.258 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ListSnap (0 ms) 2024-09-22T03:54:13.258 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-09-22T03:54:13.258 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.258 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.258 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.258 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.258 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (0 ms) 2024-09-22T03:54:13.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-09-22T03:54:13.259 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.259 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.259 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.259 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlush (0 ms) 2024-09-22T03:54:13.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.Flush 2024-09-22T03:54:13.259 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.259 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.259 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.259 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Flush (0 ms) 2024-09-22T03:54:13.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-09-22T03:54:13.259 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.260 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.260 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.260 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushSnap (0 ms) 2024-09-22T03:54:13.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-09-22T03:54:13.260 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.260 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.260 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.260 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushTryFlushRaces (0 ms) 2024-09-22T03:54:13.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-09-22T03:54:13.260 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.260 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.260 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.260 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlushReadRace (0 ms) 2024-09-22T03:54:13.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-09-22T03:54:13.261 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.261 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.261 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.261 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetRead (0 ms) 2024-09-22T03:54:13.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-09-22T03:54:13.261 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.261 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.261 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.261 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetWrite (1 ms) 2024-09-22T03:54:13.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-09-22T03:54:13.262 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.262 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.262 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.262 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetTrim (0 ms) 2024-09-22T03:54:13.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-09-22T03:54:13.262 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.262 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.262 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.262 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteOn2ndRead (0 ms) 2024-09-22T03:54:13.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-09-22T03:54:13.263 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.263 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.263 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.263 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ProxyRead (0 ms) 2024-09-22T03:54:13.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.CachePin 2024-09-22T03:54:13.263 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.263 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.263 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.263 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.CachePin (0 ms) 2024-09-22T03:54:13.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-09-22T03:54:13.263 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.263 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.264 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.264 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.SetRedirectRead (0 ms) 2024-09-22T03:54:13.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-09-22T03:54:13.264 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.264 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.264 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.264 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestPromoteRead (0 ms) 2024-09-22T03:54:13.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-09-22T03:54:13.264 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.264 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.264 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.264 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.265 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRefRead (0 ms) 2024-09-22T03:54:13.265 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-09-22T03:54:13.265 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.265 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.265 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.265 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.265 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestUnset (0 ms) 2024-09-22T03:54:13.265 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-09-22T03:54:13.265 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.265 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.265 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.265 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.265 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestDedupRefRead (0 ms) 2024-09-22T03:54:13.265 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-09-22T03:54:13.266 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.266 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.266 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.266 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.266 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount (0 ms) 2024-09-22T03:54:13.266 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-09-22T03:54:13.266 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.266 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.266 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.266 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.266 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (0 ms) 2024-09-22T03:54:13.266 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-09-22T03:54:13.266 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.266 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.267 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.267 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.267 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (0 ms) 2024-09-22T03:54:13.267 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-09-22T03:54:13.267 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.267 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.267 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.267 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.267 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (0 ms) 2024-09-22T03:54:13.267 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-09-22T03:54:13.267 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.267 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.267 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.267 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.267 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (0 ms) 2024-09-22T03:54:13.268 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-09-22T03:54:13.268 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.268 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.268 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.268 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.268 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapIncCount (0 ms) 2024-09-22T03:54:13.268 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-09-22T03:54:13.268 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.268 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.268 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.268 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.268 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvict (0 ms) 2024-09-22T03:54:13.268 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-09-22T03:54:13.268 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.269 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.269 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.269 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.269 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictPromote (0 ms) 2024-09-22T03:54:13.269 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-09-22T03:54:13.269 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.269 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.269 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.269 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.269 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (0 ms) 2024-09-22T03:54:13.269 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-09-22T03:54:13.269 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.270 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.270 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.270 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.270 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.DedupFlushRead (0 ms) 2024-09-22T03:54:13.270 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-09-22T03:54:13.270 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.270 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.270 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.270 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.270 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushSnap (0 ms) 2024-09-22T03:54:13.270 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-09-22T03:54:13.270 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.270 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.271 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.271 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.271 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushDupCount (0 ms) 2024-09-22T03:54:13.271 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-09-22T03:54:13.271 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.271 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.271 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.271 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.271 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringFlush (0 ms) 2024-09-22T03:54:13.271 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-09-22T03:54:13.271 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.271 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.271 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.271 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.272 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (0 ms) 2024-09-22T03:54:13.272 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-09-22T03:54:13.272 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.272 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.272 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.272 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.272 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollback (0 ms) 2024-09-22T03:54:13.272 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-09-22T03:54:13.272 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.272 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.272 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.272 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.272 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (0 ms) 2024-09-22T03:54:13.272 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-09-22T03:54:13.273 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.273 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.273 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.273 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.273 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictRollback (0 ms) 2024-09-22T03:54:13.273 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-09-22T03:54:13.273 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.273 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.273 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.273 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.273 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PropagateBaseTierError (0 ms) 2024-09-22T03:54:13.273 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-09-22T03:54:13.274 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.274 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.274 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.274 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.274 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HelloWriteReturn (0 ms) 2024-09-22T03:54:13.274 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-09-22T03:54:13.274 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:223: Skipped 2024-09-22T03:54:13.274 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.274 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:233: Skipped 2024-09-22T03:54:13.274 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:13.274 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (1 ms) 2024-09-22T03:54:14.245 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 48 tests from LibRadosTwoPoolsPP (2 ms total) 2024-09-22T03:54:14.245 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:54:14.245 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosTierECPP 2024-09-22T03:54:14.245 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:375: Skipped 2024-09-22T03:54:14.245 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.245 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTierECPP.Dirty 2024-09-22T03:54:14.245 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:54:14.246 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.246 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:54:14.246 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.246 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTierECPP.Dirty (0 ms) 2024-09-22T03:54:14.246 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-09-22T03:54:14.246 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:54:14.246 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.246 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:54:14.246 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.246 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTierECPP.FlushWriteRaces (0 ms) 2024-09-22T03:54:14.246 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTierECPP.CallForcesPromote 2024-09-22T03:54:14.246 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:54:14.246 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.246 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:54:14.247 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.247 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTierECPP.CallForcesPromote (0 ms) 2024-09-22T03:54:14.247 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTierECPP.HitSetNone 2024-09-22T03:54:14.247 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:54:14.247 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.247 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:54:14.247 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.247 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTierECPP.HitSetNone (0 ms) 2024-09-22T03:54:14.247 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:383: Skipped 2024-09-22T03:54:14.247 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.247 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosTierECPP (0 ms total) 2024-09-22T03:54:14.247 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:54:14.247 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 22 tests from LibRadosTwoPoolsECPP 2024-09-22T03:54:14.247 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6267: Skipped 2024-09-22T03:54:14.248 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.248 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-09-22T03:54:14.248 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.248 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.249 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.249 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.249 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Overlay (0 ms) 2024-09-22T03:54:14.249 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.Promote 2024-09-22T03:54:14.249 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.249 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.250 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.250 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.250 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Promote (0 ms) 2024-09-22T03:54:14.250 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-09-22T03:54:14.250 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.250 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.250 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.250 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.250 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnap (1 ms) 2024-09-22T03:54:14.250 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-09-22T03:54:14.250 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.250 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.250 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.250 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.251 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (0 ms) 2024-09-22T03:54:14.251 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-09-22T03:54:14.251 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.251 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.251 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.251 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.251 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Whiteout (0 ms) 2024-09-22T03:54:14.251 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.Evict 2024-09-22T03:54:14.251 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.251 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.251 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.251 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.251 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Evict (0 ms) 2024-09-22T03:54:14.252 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-09-22T03:54:14.252 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.252 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.252 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.252 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.252 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.EvictSnap (0 ms) 2024-09-22T03:54:14.252 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-09-22T03:54:14.252 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.252 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.252 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.252 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.252 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlush (0 ms) 2024-09-22T03:54:14.252 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-09-22T03:54:14.252 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.253 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.253 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.253 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.253 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FailedFlush (0 ms) 2024-09-22T03:54:14.253 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.Flush 2024-09-22T03:54:14.253 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.253 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.253 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.253 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.253 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Flush (0 ms) 2024-09-22T03:54:14.253 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-09-22T03:54:14.253 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.253 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.253 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.254 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.254 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushSnap (0 ms) 2024-09-22T03:54:14.254 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-09-22T03:54:14.254 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.254 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.254 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.254 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.254 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (0 ms) 2024-09-22T03:54:14.254 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-09-22T03:54:14.254 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.254 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.254 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.254 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlushReadRace (0 ms) 2024-09-22T03:54:14.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-09-22T03:54:14.255 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.255 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.255 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.255 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetRead (0 ms) 2024-09-22T03:54:14.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-09-22T03:54:14.255 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.255 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.255 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.255 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetTrim (0 ms) 2024-09-22T03:54:14.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-09-22T03:54:14.256 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.256 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.256 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.256 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (0 ms) 2024-09-22T03:54:14.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-09-22T03:54:14.256 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.256 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.256 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.256 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ProxyRead (0 ms) 2024-09-22T03:54:14.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-09-22T03:54:14.256 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.256 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.257 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.257 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.257 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CachePin (0 ms) 2024-09-22T03:54:14.257 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-09-22T03:54:14.257 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.257 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.257 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.257 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.257 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetRedirectRead (0 ms) 2024-09-22T03:54:14.257 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-09-22T03:54:14.257 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.257 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.257 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.257 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.258 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetChunkRead (0 ms) 2024-09-22T03:54:14.258 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-09-22T03:54:14.258 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.258 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.258 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.258 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.258 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ManifestPromoteRead (0 ms) 2024-09-22T03:54:14.258 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-09-22T03:54:14.258 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6278: Skipped 2024-09-22T03:54:14.258 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.258 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6288: Skipped 2024-09-22T03:54:14.258 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.258 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TrySetDedupTier (0 ms) 2024-09-22T03:54:14.259 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/tier_cxx.cc:6272: Skipped 2024-09-22T03:54:14.259 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:14.259 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 22 tests from LibRadosTwoPoolsECPP (1 ms total) 2024-09-22T03:54:14.259 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:54:14.259 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:54:14.259 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 77 tests from 4 test suites ran. (5894 ms total) 2024-09-22T03:54:14.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 0 tests. 2024-09-22T03:54:14.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] 77 tests, listed below: 2024-09-22T03:54:14.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTierPP.Dirty 2024-09-22T03:54:14.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTierPP.FlushWriteRaces 2024-09-22T03:54:14.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTierPP.HitSetNone 2024-09-22T03:54:14.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Overlay 2024-09-22T03:54:14.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Promote 2024-09-22T03:54:14.259 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnap 2024-09-22T03:54:14.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-09-22T03:54:14.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-09-22T03:54:14.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Whiteout 2024-09-22T03:54:14.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-09-22T03:54:14.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Evict 2024-09-22T03:54:14.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap 2024-09-22T03:54:14.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnap2 2024-09-22T03:54:14.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ListSnap 2024-09-22T03:54:14.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-09-22T03:54:14.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlush 2024-09-22T03:54:14.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.Flush 2024-09-22T03:54:14.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushSnap 2024-09-22T03:54:14.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-09-22T03:54:14.260 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-09-22T03:54:14.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetRead 2024-09-22T03:54:14.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetWrite 2024-09-22T03:54:14.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HitSetTrim 2024-09-22T03:54:14.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-09-22T03:54:14.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ProxyRead 2024-09-22T03:54:14.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.CachePin 2024-09-22T03:54:14.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.SetRedirectRead 2024-09-22T03:54:14.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-09-22T03:54:14.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRefRead 2024-09-22T03:54:14.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestUnset 2024-09-22T03:54:14.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-09-22T03:54:14.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-09-22T03:54:14.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-09-22T03:54:14.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-09-22T03:54:14.261 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-09-22T03:54:14.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-09-22T03:54:14.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-09-22T03:54:14.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvict 2024-09-22T03:54:14.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-09-22T03:54:14.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-09-22T03:54:14.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.DedupFlushRead 2024-09-22T03:54:14.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-09-22T03:54:14.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-09-22T03:54:14.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-09-22T03:54:14.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-09-22T03:54:14.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollback 2024-09-22T03:54:14.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-09-22T03:54:14.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-09-22T03:54:14.262 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-09-22T03:54:14.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-09-22T03:54:14.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-09-22T03:54:14.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTierECPP.Dirty 2024-09-22T03:54:14.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTierECPP.FlushWriteRaces 2024-09-22T03:54:14.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTierECPP.CallForcesPromote 2024-09-22T03:54:14.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTierECPP.HitSetNone 2024-09-22T03:54:14.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Overlay 2024-09-22T03:54:14.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Promote 2024-09-22T03:54:14.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnap 2024-09-22T03:54:14.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-09-22T03:54:14.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Whiteout 2024-09-22T03:54:14.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Evict 2024-09-22T03:54:14.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.EvictSnap 2024-09-22T03:54:14.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlush 2024-09-22T03:54:14.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FailedFlush 2024-09-22T03:54:14.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.Flush 2024-09-22T03:54:14.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushSnap 2024-09-22T03:54:14.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-09-22T03:54:14.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-09-22T03:54:14.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetRead 2024-09-22T03:54:14.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.HitSetTrim 2024-09-22T03:54:14.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-09-22T03:54:14.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ProxyRead 2024-09-22T03:54:14.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.CachePin 2024-09-22T03:54:14.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-09-22T03:54:14.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.SetChunkRead 2024-09-22T03:54:14.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-09-22T03:54:14.264 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-09-22T03:54:14.265 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:54:14.265 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:54:14.265 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_pool 2024-09-22T03:54:14.269 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:54:14.270 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 9 tests from 1 test suite. 2024-09-22T03:54:14.270 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:54:14.270 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 9 tests from LibRadosPools 2024-09-22T03:54:14.270 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosPools.PoolList 2024-09-22T03:54:17.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosPools.PoolList (2986 ms) 2024-09-22T03:54:17.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosPools.PoolLookup 2024-09-22T03:54:20.281 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosPools.PoolLookup (3026 ms) 2024-09-22T03:54:20.281 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosPools.PoolLookup2 2024-09-22T03:54:23.245 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosPools.PoolLookup2 (2963 ms) 2024-09-22T03:54:23.245 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-09-22T03:54:26.304 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosPools.PoolLookupOtherInstance (3060 ms) 2024-09-22T03:54:26.305 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-09-22T03:54:29.316 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3012 ms) 2024-09-22T03:54:29.317 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosPools.PoolDelete 2024-09-22T03:54:34.250 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosPools.PoolDelete (4934 ms) 2024-09-22T03:54:34.250 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosPools.PoolCreateDelete 2024-09-22T03:54:39.271 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosPools.PoolCreateDelete (5021 ms) 2024-09-22T03:54:39.271 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-09-22T03:54:44.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosPools.PoolCreateWithCrushRule (4983 ms) 2024-09-22T03:54:44.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosPools.PoolGetBaseTier 2024-09-22T03:54:44.255 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/pool.cc:139: Skipped 2024-09-22T03:54:44.255 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:54:44.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosPools.PoolGetBaseTier (0 ms) 2024-09-22T03:54:44.255 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 9 tests from LibRadosPools (29985 ms total) 2024-09-22T03:54:44.255 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:54:44.255 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:54:44.255 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 9 tests from 1 test suite ran. (29985 ms total) 2024-09-22T03:54:44.255 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 8 tests. 2024-09-22T03:54:44.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] 1 test, listed below: 2024-09-22T03:54:44.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosPools.PoolGetBaseTier 2024-09-22T03:54:44.257 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:54:44.257 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:54:44.258 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_snapshots 2024-09-22T03:54:44.279 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:54:44.279 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 13 tests from 4 test suites. 2024-09-22T03:54:44.279 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:54:44.280 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosSnapshots 2024-09-22T03:54:46.299 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshots.SnapList 2024-09-22T03:54:48.300 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshots.SnapList (2002 ms) 2024-09-22T03:54:48.300 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshots.SnapRemove 2024-09-22T03:54:50.325 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshots.SnapRemove (2025 ms) 2024-09-22T03:54:50.325 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshots.Rollback 2024-09-22T03:54:52.335 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshots.Rollback (2010 ms) 2024-09-22T03:54:52.335 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshots.SnapGetName 2024-09-22T03:54:54.254 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshots.SnapGetName (1919 ms) 2024-09-22T03:54:55.258 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosSnapshots (7956 ms total) 2024-09-22T03:54:55.258 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:54:55.258 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-09-22T03:54:57.269 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-09-22T03:55:01.370 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshotsSelfManaged.Snap (4101 ms) 2024-09-22T03:55:01.371 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-09-22T03:55:05.386 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshotsSelfManaged.Rollback (4016 ms) 2024-09-22T03:55:05.387 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-09-22T03:55:10.432 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (5046 ms) 2024-09-22T03:55:11.357 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManaged (13163 ms total) 2024-09-22T03:55:11.357 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:55:11.357 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosSnapshotsEC 2024-09-22T03:55:11.357 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:168: Skipped 2024-09-22T03:55:11.358 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:55:11.358 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsEC.SnapList 2024-09-22T03:55:11.358 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:55:11.358 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:55:11.358 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:55:11.358 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:55:11.358 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapList (0 ms) 2024-09-22T03:55:11.358 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-09-22T03:55:11.358 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:55:11.358 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:55:11.358 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:55:11.358 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:55:11.358 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapRemove (0 ms) 2024-09-22T03:55:11.358 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsEC.Rollback 2024-09-22T03:55:11.359 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:55:11.359 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:55:11.359 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:55:11.359 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:55:11.359 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsEC.Rollback (0 ms) 2024-09-22T03:55:11.359 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-09-22T03:55:11.359 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:55:11.359 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:55:11.359 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:55:11.359 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:55:11.359 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapGetName (0 ms) 2024-09-22T03:55:11.359 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:176: Skipped 2024-09-22T03:55:11.359 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:55:11.359 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosSnapshotsEC (0 ms total) 2024-09-22T03:55:11.360 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:55:11.360 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-09-22T03:55:11.360 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:168: Skipped 2024-09-22T03:55:11.360 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:55:11.360 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-09-22T03:55:11.360 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:55:11.360 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:55:11.360 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:55:11.360 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:55:11.360 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Snap (0 ms) 2024-09-22T03:55:11.360 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-09-22T03:55:11.360 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:55:11.360 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:55:11.360 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:55:11.360 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:55:11.361 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Rollback (0 ms) 2024-09-22T03:55:11.361 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:176: Skipped 2024-09-22T03:55:11.361 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:55:11.361 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 2 tests from LibRadosSnapshotsSelfManagedEC (0 ms total) 2024-09-22T03:55:11.361 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:55:11.361 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:55:11.361 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 13 tests from 4 test suites ran. (27076 ms total) 2024-09-22T03:55:11.361 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 7 tests. 2024-09-22T03:55:11.361 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] 6 tests, listed below: 2024-09-22T03:55:11.361 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapList 2024-09-22T03:55:11.361 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapRemove 2024-09-22T03:55:11.361 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsEC.Rollback 2024-09-22T03:55:11.361 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsEC.SnapGetName 2024-09-22T03:55:11.361 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Snap 2024-09-22T03:55:11.362 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-09-22T03:55:11.362 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:55:11.362 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:55:11.362 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_snapshots_pp 2024-09-22T03:55:11.382 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:55:11.382 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 21 tests from 5 test suites. 2024-09-22T03:55:11.382 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:55:11.382 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 5 tests from LibRadosSnapshotsPP 2024-09-22T03:55:11.383 INFO:tasks.workunit.client.0.smithi003.stdout:seed 93034 2024-09-22T03:55:13.256 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-09-22T03:55:15.363 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshotsPP.SnapListPP (2107 ms) 2024-09-22T03:55:15.363 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-09-22T03:55:17.374 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshotsPP.SnapRemovePP (2010 ms) 2024-09-22T03:55:17.374 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-09-22T03:55:19.385 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshotsPP.RollbackPP (2011 ms) 2024-09-22T03:55:19.386 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-09-22T03:55:21.396 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2010 ms) 2024-09-22T03:55:21.396 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-09-22T03:55:25.263 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (3867 ms) 2024-09-22T03:55:26.272 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 5 tests from LibRadosSnapshotsPP (12006 ms total) 2024-09-22T03:55:26.272 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:55:26.272 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-09-22T03:55:28.274 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-09-22T03:55:32.379 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4106 ms) 2024-09-22T03:55:32.380 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-09-22T03:55:36.279 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (3900 ms) 2024-09-22T03:55:36.279 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-09-22T03:55:42.314 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (6035 ms) 2024-09-22T03:55:42.314 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-09-22T03:55:46.324 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (4010 ms) 2024-09-22T03:55:46.324 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-09-22T03:55:48.439 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2115 ms) 2024-09-22T03:55:48.439 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-09-22T03:55:48.440 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/snapshots_cxx.cc:460: Skipped 2024-09-22T03:55:48.440 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:55:48.440 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-09-22T03:55:48.440 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-09-22T03:55:50.339 INFO:tasks.workunit.client.0.smithi003.stdout:deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi003-93034-7 2024-09-22T03:55:51.340 INFO:tasks.workunit.client.0.smithi003.stdout:waiting for snaps to purge 2024-09-22T03:56:06.410 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17971 ms) 2024-09-22T03:56:07.272 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 7 tests from LibRadosSnapshotsSelfManagedPP (38137 ms total) 2024-09-22T03:56:07.272 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:56:07.272 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-09-22T03:56:07.273 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-09-22T03:56:07.279 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (7 ms) 2024-09-22T03:56:07.279 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-09-22T03:56:12.335 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5054 ms) 2024-09-22T03:56:12.335 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5061 ms total) 2024-09-22T03:56:12.335 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:56:12.335 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosSnapshotsECPP 2024-09-22T03:56:12.335 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:375: Skipped 2024-09-22T03:56:12.335 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.336 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-09-22T03:56:12.336 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:56:12.336 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.336 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:56:12.336 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.336 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapListPP (0 ms) 2024-09-22T03:56:12.336 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-09-22T03:56:12.336 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:56:12.336 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.336 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:56:12.336 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.336 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapRemovePP (0 ms) 2024-09-22T03:56:12.336 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-09-22T03:56:12.336 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:56:12.337 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.337 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:56:12.337 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.337 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.RollbackPP (0 ms) 2024-09-22T03:56:12.337 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-09-22T03:56:12.337 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:56:12.337 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.337 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:56:12.337 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.337 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapGetNamePP (0 ms) 2024-09-22T03:56:12.337 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:383: Skipped 2024-09-22T03:56:12.337 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.337 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosSnapshotsECPP (0 ms total) 2024-09-22T03:56:12.338 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:56:12.338 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-09-22T03:56:12.338 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:375: Skipped 2024-09-22T03:56:12.338 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.338 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-09-22T03:56:12.338 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:56:12.338 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.338 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:56:12.338 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.338 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.SnapPP (0 ms) 2024-09-22T03:56:12.338 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-09-22T03:56:12.338 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:56:12.338 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.339 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:56:12.339 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.339 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (0 ms) 2024-09-22T03:56:12.339 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-09-22T03:56:12.339 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:56:12.339 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.339 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:56:12.339 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.339 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (0 ms) 2024-09-22T03:56:12.339 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:383: Skipped 2024-09-22T03:56:12.339 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:12.339 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (0 ms total) 2024-09-22T03:56:12.339 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:56:12.339 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:56:12.339 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 21 tests from 5 test suites ran. (60951 ms total) 2024-09-22T03:56:12.340 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 13 tests. 2024-09-22T03:56:12.340 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] 8 tests, listed below: 2024-09-22T03:56:12.340 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-09-22T03:56:12.340 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapListPP 2024-09-22T03:56:12.340 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapRemovePP 2024-09-22T03:56:12.340 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.RollbackPP 2024-09-22T03:56:12.340 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-09-22T03:56:12.340 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-09-22T03:56:12.340 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-09-22T03:56:12.340 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-09-22T03:56:12.340 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:56:12.341 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:56:12.341 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_stat 2024-09-22T03:56:12.361 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:56:12.361 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 9 tests from 2 test suites. 2024-09-22T03:56:12.362 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:56:12.362 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 5 tests from LibRadosStat 2024-09-22T03:56:14.312 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStat.Stat 2024-09-22T03:56:14.837 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosStat.Stat (526 ms) 2024-09-22T03:56:14.837 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStat.Stat2 2024-09-22T03:56:14.897 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosStat.Stat2 (60 ms) 2024-09-22T03:56:14.897 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStat.StatNS 2024-09-22T03:56:15.001 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosStat.StatNS (104 ms) 2024-09-22T03:56:15.001 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStat.ClusterStat 2024-09-22T03:56:15.001 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosStat.ClusterStat (0 ms) 2024-09-22T03:56:15.002 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStat.PoolStat 2024-09-22T03:56:15.025 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosStat.PoolStat (23 ms) 2024-09-22T03:56:15.318 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 5 tests from LibRadosStat (713 ms total) 2024-09-22T03:56:15.318 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:56:15.318 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosStatEC 2024-09-22T03:56:15.318 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:168: Skipped 2024-09-22T03:56:15.318 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:15.318 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStatEC.Stat 2024-09-22T03:56:15.318 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:56:15.318 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:15.318 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:56:15.318 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:15.318 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosStatEC.Stat (0 ms) 2024-09-22T03:56:15.319 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStatEC.StatNS 2024-09-22T03:56:15.319 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:56:15.319 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:15.319 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:56:15.319 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:15.319 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosStatEC.StatNS (0 ms) 2024-09-22T03:56:15.319 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStatEC.ClusterStat 2024-09-22T03:56:15.319 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:56:15.319 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:15.319 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:56:15.319 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:15.319 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosStatEC.ClusterStat (0 ms) 2024-09-22T03:56:15.319 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStatEC.PoolStat 2024-09-22T03:56:15.319 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:56:15.319 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:15.320 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:56:15.320 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:15.320 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosStatEC.PoolStat (0 ms) 2024-09-22T03:56:15.320 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:176: Skipped 2024-09-22T03:56:15.320 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:15.320 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosStatEC (0 ms total) 2024-09-22T03:56:15.320 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:56:15.320 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:56:15.320 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 9 tests from 2 test suites ran. (2955 ms total) 2024-09-22T03:56:15.320 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 5 tests. 2024-09-22T03:56:15.320 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] 4 tests, listed below: 2024-09-22T03:56:15.320 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosStatEC.Stat 2024-09-22T03:56:15.320 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosStatEC.StatNS 2024-09-22T03:56:15.320 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosStatEC.ClusterStat 2024-09-22T03:56:15.321 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosStatEC.PoolStat 2024-09-22T03:56:15.321 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:56:15.321 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:56:15.321 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_stat_pp 2024-09-22T03:56:15.339 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:56:15.339 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 9 tests from 2 test suites. 2024-09-22T03:56:15.339 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:56:15.339 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 5 tests from LibRadosStatPP 2024-09-22T03:56:15.339 INFO:tasks.workunit.client.0.smithi003.stdout:seed 93168 2024-09-22T03:56:17.327 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStatPP.StatPP 2024-09-22T03:56:17.842 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosStatPP.StatPP (516 ms) 2024-09-22T03:56:17.843 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-09-22T03:56:17.886 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosStatPP.Stat2Mtime2PP (44 ms) 2024-09-22T03:56:17.887 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStatPP.ClusterStatPP 2024-09-22T03:56:17.887 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-09-22T03:56:17.887 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStatPP.PoolStatPP 2024-09-22T03:56:17.908 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosStatPP.PoolStatPP (22 ms) 2024-09-22T03:56:17.908 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStatPP.StatPPNS 2024-09-22T03:56:18.003 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosStatPP.StatPPNS (95 ms) 2024-09-22T03:56:18.272 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 5 tests from LibRadosStatPP (677 ms total) 2024-09-22T03:56:18.272 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:56:18.272 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosStatECPP 2024-09-22T03:56:18.272 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:375: Skipped 2024-09-22T03:56:18.272 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:18.272 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStatECPP.StatPP 2024-09-22T03:56:18.272 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:56:18.272 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:18.272 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:56:18.272 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:18.272 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosStatECPP.StatPP (0 ms) 2024-09-22T03:56:18.272 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStatECPP.ClusterStatPP 2024-09-22T03:56:18.272 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:56:18.273 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:18.273 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:56:18.273 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:18.273 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosStatECPP.ClusterStatPP (0 ms) 2024-09-22T03:56:18.273 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStatECPP.PoolStatPP 2024-09-22T03:56:18.273 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:56:18.273 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:18.273 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:56:18.273 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:18.273 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosStatECPP.PoolStatPP (0 ms) 2024-09-22T03:56:18.273 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosStatECPP.StatPPNS 2024-09-22T03:56:18.273 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:56:18.273 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:18.273 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:56:18.273 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:18.274 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosStatECPP.StatPPNS (0 ms) 2024-09-22T03:56:18.274 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:383: Skipped 2024-09-22T03:56:18.274 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:18.274 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosStatECPP (0 ms total) 2024-09-22T03:56:18.274 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:56:18.274 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:56:18.274 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 9 tests from 2 test suites ran. (2931 ms total) 2024-09-22T03:56:18.274 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 5 tests. 2024-09-22T03:56:18.274 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] 4 tests, listed below: 2024-09-22T03:56:18.274 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosStatECPP.StatPP 2024-09-22T03:56:18.274 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosStatECPP.ClusterStatPP 2024-09-22T03:56:18.274 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosStatECPP.PoolStatPP 2024-09-22T03:56:18.274 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosStatECPP.StatPPNS 2024-09-22T03:56:18.275 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:56:18.275 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:56:18.275 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_watch_notify 2024-09-22T03:56:18.293 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:56:18.293 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 11 tests from 2 test suites. 2024-09-22T03:56:18.293 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:56:18.293 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 10 tests from LibRadosWatchNotify 2024-09-22T03:56:20.343 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify 2024-09-22T03:56:20.695 INFO:tasks.workunit.client.0.smithi003.stdout:watch_notify_test_cb 2024-09-22T03:56:20.756 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotify.WatchNotify (413 ms) 2024-09-22T03:56:20.756 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotify.Watch2Delete 2024-09-22T03:56:20.819 INFO:tasks.workunit.client.0.smithi003.stdout:waiting up to 300 for disconnect notification ... 2024-09-22T03:56:23.341 INFO:tasks.workunit.client.0.smithi003.stdout:watch_notify2_test_errcb cookie 9372512 err -107 2024-09-22T03:56:23.832 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotify.Watch2Delete (3077 ms) 2024-09-22T03:56:23.832 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-09-22T03:56:23.908 INFO:tasks.workunit.client.0.smithi003.stdout:waiting up to 300 for disconnect notification ... 2024-09-22T03:56:28.343 INFO:tasks.workunit.client.0.smithi003.stdout:watch_notify2_test_errcb cookie 9372512 err -107 2024-09-22T03:56:28.919 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotify.AioWatchDelete (5087 ms) 2024-09-22T03:56:28.919 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify2 2024-09-22T03:56:28.988 INFO:tasks.workunit.client.0.smithi003.stdout:watch_notify2_test_cb from 15553 notify_id 1752346656768 cookie 9384176 2024-09-22T03:56:29.016 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotify.WatchNotify2 (96 ms) 2024-09-22T03:56:29.016 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-09-22T03:56:29.061 INFO:tasks.workunit.client.0.smithi003.stdout:watch_notify2_test_cb from 15553 notify_id 1752346656768 cookie 9404112 2024-09-22T03:56:29.093 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotify.AioWatchNotify2 (77 ms) 2024-09-22T03:56:29.093 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotify.AioNotify 2024-09-22T03:56:29.140 INFO:tasks.workunit.client.0.smithi003.stdout:watch_notify2_test_cb from 15553 notify_id 1752346656769 cookie 9416192 2024-09-22T03:56:29.167 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotify.AioNotify (74 ms) 2024-09-22T03:56:29.167 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-09-22T03:56:29.229 INFO:tasks.workunit.client.0.smithi003.stdout:watch_notify2_test_cb from 15553 notify_id 1752346656770 cookie 9416192 2024-09-22T03:56:29.229 INFO:tasks.workunit.client.0.smithi003.stdout:watch_notify2_test_cb from 15553 notify_id 1752346656770 cookie 9417312 2024-09-22T03:56:29.270 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotify.WatchNotify2Multi (104 ms) 2024-09-22T03:56:29.270 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-09-22T03:56:29.313 INFO:tasks.workunit.client.0.smithi003.stdout:watch_notify2_test_cb from 15553 notify_id 1752346656771 cookie 9416192 2024-09-22T03:56:32.314 INFO:tasks.workunit.client.0.smithi003.stdout:watch_notify2_test_cb from 15553 notify_id 1752346656772 cookie 9416192 2024-09-22T03:56:32.345 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3074 ms) 2024-09-22T03:56:32.345 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-09-22T03:56:32.393 INFO:tasks.workunit.client.0.smithi003.stdout:waiting up to 1024 for osd to time us out ... 2024-09-22T03:56:36.411 INFO:tasks.workunit.client.0.smithi003.stdout:watch_notify2_test_errcb cookie 9416192 err -107 2024-09-22T03:56:37.433 INFO:tasks.workunit.client.0.smithi003.stdout:watch_notify2_test_cb from 15553 notify_id 1752346656774 cookie 9416192 2024-09-22T03:56:37.470 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotify.Watch3Timeout (5126 ms) 2024-09-22T03:56:37.470 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-09-22T03:56:37.541 INFO:tasks.workunit.client.0.smithi003.stdout:waiting up to 30 for disconnect notification ... 2024-09-22T03:56:38.346 INFO:tasks.workunit.client.0.smithi003.stdout:watch_notify2_test_errcb cookie 9416192 err -107 2024-09-22T03:56:38.568 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotify.AioWatchDelete2 (1098 ms) 2024-09-22T03:56:39.294 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 10 tests from LibRadosWatchNotify (18226 ms total) 2024-09-22T03:56:39.294 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:56:39.294 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibRadosWatchNotifyEC 2024-09-22T03:56:39.294 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:168: Skipped 2024-09-22T03:56:39.294 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:39.294 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-09-22T03:56:39.294 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:182: Skipped 2024-09-22T03:56:39.294 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:39.294 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:196: Skipped 2024-09-22T03:56:39.294 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:39.295 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosWatchNotifyEC.WatchNotify (0 ms) 2024-09-22T03:56:39.295 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/TestCase.cc:176: Skipped 2024-09-22T03:56:39.295 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:39.295 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibRadosWatchNotifyEC (0 ms total) 2024-09-22T03:56:39.295 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:56:39.295 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:56:39.295 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 11 tests from 2 test suites ran. (21000 ms total) 2024-09-22T03:56:39.295 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 10 tests. 2024-09-22T03:56:39.295 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] 1 test, listed below: 2024-09-22T03:56:39.295 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosWatchNotifyEC.WatchNotify 2024-09-22T03:56:39.297 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:56:39.297 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:56:39.297 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_watch_notify_pp 2024-09-22T03:56:39.321 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:56:39.321 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 16 tests from 2 test suites. 2024-09-22T03:56:39.321 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:56:39.321 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 2 tests from LibRadosWatchNotifyECPP 2024-09-22T03:56:39.321 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:375: Skipped 2024-09-22T03:56:39.321 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:39.321 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-09-22T03:56:39.321 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:56:39.321 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:39.321 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:56:39.321 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:39.321 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotify (0 ms) 2024-09-22T03:56:39.321 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-09-22T03:56:39.322 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:389: Skipped 2024-09-22T03:56:39.322 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:39.322 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:402: Skipped 2024-09-22T03:56:39.322 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:39.322 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (0 ms) 2024-09-22T03:56:39.322 INFO:tasks.workunit.client.0.smithi003.stdout:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5092-g62da54b4/rpm/el9/BUILD/ceph-19.3.0-5092-g62da54b4/src/test/librados/testcase_cxx.cc:383: Skipped 2024-09-22T03:56:39.322 INFO:tasks.workunit.client.0.smithi003.stdout:Not supported by crimson yet. Skipped 2024-09-22T03:56:39.322 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 2 tests from LibRadosWatchNotifyECPP (0 ms total) 2024-09-22T03:56:39.322 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:56:39.322 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-09-22T03:56:41.301 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-09-22T03:56:41.805 INFO:tasks.workunit.client.0.smithi003.stdout:notify 2024-09-22T03:56:41.826 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (525 ms) 2024-09-22T03:56:41.826 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-09-22T03:56:41.877 INFO:tasks.workunit.client.0.smithi003.stdout:notify 2024-09-22T03:56:41.897 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (71 ms) 2024-09-22T03:56:41.897 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-09-22T03:56:41.958 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (61 ms) 2024-09-22T03:56:41.958 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-09-22T03:56:42.024 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (66 ms) 2024-09-22T03:56:42.024 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-09-22T03:56:42.072 INFO:tasks.workunit.client.0.smithi003.stdout:handle_notify cookie 19821104 notify_id 1765231558657 notifier_gid 15556 2024-09-22T03:56:42.093 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (69 ms) 2024-09-22T03:56:42.093 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-09-22T03:56:42.139 INFO:tasks.workunit.client.0.smithi003.stdout:handle_notify cookie 19835808 notify_id 1765231558658 notifier_gid 15556 2024-09-22T03:56:42.159 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (65 ms) 2024-09-22T03:56:42.159 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-09-22T03:56:42.202 INFO:tasks.workunit.client.0.smithi003.stdout:handle_notify cookie 19835744 notify_id 1765231558663 notifier_gid 15556 2024-09-22T03:56:42.222 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (64 ms) 2024-09-22T03:56:42.222 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-09-22T03:56:42.271 INFO:tasks.workunit.client.0.smithi003.stdout:handle_notify cookie 19835744 notify_id 1765231558658 notifier_gid 15556 2024-09-22T03:56:42.292 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (70 ms) 2024-09-22T03:56:42.292 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-09-22T03:56:42.384 INFO:tasks.workunit.client.0.smithi003.stdout:handle_notify cookie 19835744 notify_id 1765231558659 notifier_gid 15556 2024-09-22T03:56:42.404 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (112 ms) 2024-09-22T03:56:42.405 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-09-22T03:56:42.454 INFO:tasks.workunit.client.0.smithi003.stdout:handle_notify cookie 19835744 notify_id 1765231558657 notifier_gid 15556 2024-09-22T03:56:42.474 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (70 ms) 2024-09-22T03:56:42.474 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-09-22T03:56:42.524 INFO:tasks.workunit.client.0.smithi003.stdout: trying... 2024-09-22T03:56:42.527 INFO:tasks.workunit.client.0.smithi003.stdout:handle_notify cookie 19835744 notify_id 1765231558659 notifier_gid 15556 2024-09-22T03:56:43.527 INFO:tasks.workunit.client.0.smithi003.stdout: timed out 2024-09-22T03:56:43.550 INFO:tasks.workunit.client.0.smithi003.stdout: flushing 2024-09-22T03:56:45.528 INFO:tasks.workunit.client.0.smithi003.stdout: flushed 2024-09-22T03:56:45.528 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3054 ms) 2024-09-22T03:56:45.528 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-09-22T03:56:45.579 INFO:tasks.workunit.client.0.smithi003.stdout: trying... 2024-09-22T03:56:45.583 INFO:tasks.workunit.client.0.smithi003.stdout:handle_notify cookie 19835744 notify_id 1765231558660 notifier_gid 15556 2024-09-22T03:56:46.581 INFO:tasks.workunit.client.0.smithi003.stdout: timed out 2024-09-22T03:56:46.600 INFO:tasks.workunit.client.0.smithi003.stdout: flushing 2024-09-22T03:56:48.583 INFO:tasks.workunit.client.0.smithi003.stdout: flushed 2024-09-22T03:56:48.583 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3055 ms) 2024-09-22T03:56:48.583 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-09-22T03:56:48.641 INFO:tasks.workunit.client.0.smithi003.stdout:List watches 2024-09-22T03:56:48.641 INFO:tasks.workunit.client.0.smithi003.stdout:notify2 2024-09-22T03:56:48.645 INFO:tasks.workunit.client.0.smithi003.stdout:handle_notify cookie 19835744 notify_id 1765231558661 notifier_gid 15556 2024-09-22T03:56:51.651 INFO:tasks.workunit.client.0.smithi003.stdout:notify2 done 2024-09-22T03:56:51.652 INFO:tasks.workunit.client.0.smithi003.stdout:watch_check 2024-09-22T03:56:51.652 INFO:tasks.workunit.client.0.smithi003.stdout:unwatch2 2024-09-22T03:56:51.675 INFO:tasks.workunit.client.0.smithi003.stdout: flushing 2024-09-22T03:56:51.675 INFO:tasks.workunit.client.0.smithi003.stdout:done 2024-09-22T03:56:51.675 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3092 ms) 2024-09-22T03:56:51.675 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-09-22T03:56:51.730 INFO:tasks.workunit.client.0.smithi003.stdout:List watches 2024-09-22T03:56:51.730 INFO:tasks.workunit.client.0.smithi003.stdout:notify2 2024-09-22T03:56:51.734 INFO:tasks.workunit.client.0.smithi003.stdout:handle_notify cookie 19839168 notify_id 1769526525958 notifier_gid 15556 2024-09-22T03:56:54.740 INFO:tasks.workunit.client.0.smithi003.stdout:notify2 done 2024-09-22T03:56:54.740 INFO:tasks.workunit.client.0.smithi003.stdout:watch_check 2024-09-22T03:56:54.740 INFO:tasks.workunit.client.0.smithi003.stdout:unwatch2 2024-09-22T03:56:54.763 INFO:tasks.workunit.client.0.smithi003.stdout: flushing 2024-09-22T03:56:54.763 INFO:tasks.workunit.client.0.smithi003.stdout:done 2024-09-22T03:56:54.763 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3087 ms) 2024-09-22T03:56:55.284 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (13461 ms total) 2024-09-22T03:56:55.285 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:56:55.285 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:56:55.285 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 16 tests from 2 test suites ran. (15964 ms total) 2024-09-22T03:56:55.285 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 14 tests. 2024-09-22T03:56:55.285 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] 2 tests, listed below: 2024-09-22T03:56:55.285 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotify 2024-09-22T03:56:55.285 INFO:tasks.workunit.client.0.smithi003.stdout:[ SKIPPED ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-09-22T03:56:55.287 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:56:55.287 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:56:55.287 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_cmd 2024-09-22T03:56:55.310 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:56:55.310 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 4 tests from 1 test suite. 2024-09-22T03:56:55.310 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:56:55.310 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosCmd 2024-09-22T03:56:55.310 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosCmd.MonDescribe 2024-09-22T03:56:55.354 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosCmd.MonDescribe (44 ms) 2024-09-22T03:56:55.354 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosCmd.OSDCmd 2024-09-22T03:56:55.539 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosCmd.OSDCmd (185 ms) 2024-09-22T03:56:55.539 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosCmd.PGCmd 2024-09-22T03:56:58.303 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosCmd.PGCmd (2763 ms) 2024-09-22T03:56:58.303 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosCmd.WatchLog 2024-09-22T03:56:58.342 INFO:tasks.workunit.client.0.smithi003.stdout:got: 2024-09-22T03:56:57.287782+0000 mon.a [INF] from='client.? 172.21.15.3:0/2421493231' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi003-93227-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-22T03:56:59.299 INFO:tasks.workunit.client.0.smithi003.stdout:got: 2024-09-22T03:56:58.341783+0000 client.admin [INF] onexx 2024-09-22T03:56:59.299 INFO:tasks.workunit.client.0.smithi003.stdout:got: 2024-09-22T03:56:58.341954+0000 mon.a [INF] from='client.? 172.21.15.3:0/2563781891' entity='client.admin' cmd={"prefix":"log", "logtext":["onexx"]} : dispatch 2024-09-22T03:57:03.305 INFO:tasks.workunit.client.0.smithi003.stdout:got: 2024-09-22T03:57:01.303826+0000 mon.a [INF] from='client.? 172.21.15.3:0/2563781891' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-22T03:57:04.285 INFO:tasks.workunit.client.0.smithi003.stdout:got: 2024-09-22T03:57:03.304656+0000 client.admin [INF] threexx 2024-09-22T03:57:04.285 INFO:tasks.workunit.client.0.smithi003.stdout:got: 2024-09-22T03:57:03.304968+0000 mon.a [INF] from='client.? 172.21.15.3:0/2563781891' entity='client.admin' cmd={"prefix":"log", "logtext":["threexx"]} : dispatch 2024-09-22T03:57:08.292 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosCmd.WatchLog (9989 ms) 2024-09-22T03:57:08.292 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosCmd (12981 ms total) 2024-09-22T03:57:08.292 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:57:08.292 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:57:08.292 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 4 tests from 1 test suite ran. (12981 ms total) 2024-09-22T03:57:08.292 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 4 tests. 2024-09-22T03:57:08.294 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:57:08.294 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:57:08.294 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_cmd_pp 2024-09-22T03:57:08.316 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:57:08.316 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 3 tests from 1 test suite. 2024-09-22T03:57:08.316 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:57:08.317 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 3 tests from LibRadosCmd 2024-09-22T03:57:08.317 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosCmd.MonDescribePP 2024-09-22T03:57:08.354 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosCmd.MonDescribePP (38 ms) 2024-09-22T03:57:08.354 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosCmd.OSDCmdPP 2024-09-22T03:57:08.391 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosCmd.OSDCmdPP (37 ms) 2024-09-22T03:57:08.391 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosCmd.PGCmdPP 2024-09-22T03:57:12.321 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosCmd.PGCmdPP (3930 ms) 2024-09-22T03:57:12.321 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 3 tests from LibRadosCmd (4005 ms total) 2024-09-22T03:57:12.321 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:57:12.322 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:57:12.322 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 3 tests from 1 test suite ran. (4005 ms total) 2024-09-22T03:57:12.322 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 3 tests. 2024-09-22T03:57:12.324 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:57:12.324 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:57:12.324 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_service 2024-09-22T03:57:12.363 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:57:12.362+0000 7f77638f4840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:57:12.363 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:57:12.362+0000 7f77638f4840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:57:12.363 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:57:12.362+0000 7f77638f4840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:57:12.363 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 4 tests from 1 test suite. 2024-09-22T03:57:12.363 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:57:12.363 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosService 2024-09-22T03:57:12.364 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosService.RegisterEarly 2024-09-22T03:57:17.398 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosService.RegisterEarly (5035 ms) 2024-09-22T03:57:17.398 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosService.RegisterLate 2024-09-22T03:57:17.430 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosService.RegisterLate (32 ms) 2024-09-22T03:57:17.430 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosService.StatusFormat 2024-09-22T03:57:17.620 INFO:tasks.workunit.client.0.smithi003.stdout: cluster: 2024-09-22T03:57:17.620 INFO:tasks.workunit.client.0.smithi003.stdout: id: 012c124d-1263-4b17-8b29-e14858734658 2024-09-22T03:57:17.620 INFO:tasks.workunit.client.0.smithi003.stdout: health: HEALTH_WARN 2024-09-22T03:57:17.620 INFO:tasks.workunit.client.0.smithi003.stdout: 1 pool(s) do not have an application enabled 2024-09-22T03:57:17.620 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:57:17.620 INFO:tasks.workunit.client.0.smithi003.stdout: services: 2024-09-22T03:57:17.620 INFO:tasks.workunit.client.0.smithi003.stdout: mon: 1 daemons, quorum a (age 10m) 2024-09-22T03:57:17.620 INFO:tasks.workunit.client.0.smithi003.stdout: mgr: x(active, since 10m) 2024-09-22T03:57:17.620 INFO:tasks.workunit.client.0.smithi003.stdout: osd: 4 osds: 4 up (since 10m), 4 in (since 10m) 2024-09-22T03:57:17.620 INFO:tasks.workunit.client.0.smithi003.stdout: laundry: 1 daemon active (1 hosts) 2024-09-22T03:57:17.621 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:57:17.621 INFO:tasks.workunit.client.0.smithi003.stdout: data: 2024-09-22T03:57:17.621 INFO:tasks.workunit.client.0.smithi003.stdout: pools: 3 pools, 41 pgs 2024-09-22T03:57:17.621 INFO:tasks.workunit.client.0.smithi003.stdout: objects: 5 objects, 577 KiB 2024-09-22T03:57:17.621 INFO:tasks.workunit.client.0.smithi003.stdout: usage: 20 MiB used, 199 GiB / 200 GiB avail 2024-09-22T03:57:17.621 INFO:tasks.workunit.client.0.smithi003.stdout: pgs: 41 active+clean 2024-09-22T03:57:17.621 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:57:17.621 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:57:19.651 INFO:tasks.workunit.client.0.smithi003.stdout: cluster: 2024-09-22T03:57:19.651 INFO:tasks.workunit.client.0.smithi003.stdout: id: 012c124d-1263-4b17-8b29-e14858734658 2024-09-22T03:57:19.651 INFO:tasks.workunit.client.0.smithi003.stdout: health: HEALTH_WARN 2024-09-22T03:57:19.651 INFO:tasks.workunit.client.0.smithi003.stdout: 1 pool(s) do not have an application enabled 2024-09-22T03:57:19.651 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:57:19.651 INFO:tasks.workunit.client.0.smithi003.stdout: services: 2024-09-22T03:57:19.651 INFO:tasks.workunit.client.0.smithi003.stdout: mon: 1 daemons, quorum a (age 10m) 2024-09-22T03:57:19.651 INFO:tasks.workunit.client.0.smithi003.stdout: mgr: x(active, since 10m) 2024-09-22T03:57:19.651 INFO:tasks.workunit.client.0.smithi003.stdout: osd: 4 osds: 4 up (since 10m), 4 in (since 10m) 2024-09-22T03:57:19.652 INFO:tasks.workunit.client.0.smithi003.stdout: foo: 16 portals active (1 hosts, 3 zones) 2024-09-22T03:57:19.652 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:57:19.652 INFO:tasks.workunit.client.0.smithi003.stdout: data: 2024-09-22T03:57:19.652 INFO:tasks.workunit.client.0.smithi003.stdout: pools: 3 pools, 41 pgs 2024-09-22T03:57:19.652 INFO:tasks.workunit.client.0.smithi003.stdout: objects: 5 objects, 577 KiB 2024-09-22T03:57:19.652 INFO:tasks.workunit.client.0.smithi003.stdout: usage: 20 MiB used, 199 GiB / 200 GiB avail 2024-09-22T03:57:19.652 INFO:tasks.workunit.client.0.smithi003.stdout: pgs: 41 active+clean 2024-09-22T03:57:19.652 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:57:19.652 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:57:19.677 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosService.StatusFormat (2247 ms) 2024-09-22T03:57:19.677 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosService.Status 2024-09-22T03:57:39.708 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosService.Status (20031 ms) 2024-09-22T03:57:39.708 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosService (27345 ms total) 2024-09-22T03:57:39.709 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:57:39.709 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:57:39.709 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 4 tests from 1 test suite ran. (27345 ms total) 2024-09-22T03:57:39.709 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 4 tests. 2024-09-22T03:57:39.714 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:57:39.714 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:57:39.714 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_service_pp 2024-09-22T03:57:39.752 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:57:39.751+0000 7fcd59e32840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:57:39.752 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:57:39.751+0000 7fcd59e32840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:57:39.752 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T03:57:39.751+0000 7fcd59e32840 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T03:57:39.753 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 4 tests from 1 test suite. 2024-09-22T03:57:39.753 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:57:39.753 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosServicePP 2024-09-22T03:57:39.753 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosServicePP.RegisterEarly 2024-09-22T03:57:44.783 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosServicePP.RegisterEarly (5031 ms) 2024-09-22T03:57:44.783 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosServicePP.RegisterLate 2024-09-22T03:57:44.832 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosServicePP.RegisterLate (49 ms) 2024-09-22T03:57:44.833 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosServicePP.Status 2024-09-22T03:58:04.864 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosServicePP.Status (20032 ms) 2024-09-22T03:58:04.864 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosServicePP.Close 2024-09-22T03:58:04.864 INFO:tasks.workunit.client.0.smithi003.stdout:attempt 0 of 20 2024-09-22T03:58:09.927 INFO:tasks.workunit.client.0.smithi003.stdout: failed to deregister: 2024-09-22T03:58:09.927 INFO:tasks.workunit.client.0.smithi003.stdout:{ 2024-09-22T03:58:09.927 INFO:tasks.workunit.client.0.smithi003.stdout: "epoch": 9, 2024-09-22T03:58:09.927 INFO:tasks.workunit.client.0.smithi003.stdout: "modified": "2024-09-22T03:58:05.713339+0000", 2024-09-22T03:58:09.927 INFO:tasks.workunit.client.0.smithi003.stdout: "services": { 2024-09-22T03:58:09.927 INFO:tasks.workunit.client.0.smithi003.stdout: "laundry": { 2024-09-22T03:58:09.927 INFO:tasks.workunit.client.0.smithi003.stdout: "daemons": { 2024-09-22T03:58:09.927 INFO:tasks.workunit.client.0.smithi003.stdout: "summary": "", 2024-09-22T03:58:09.928 INFO:tasks.workunit.client.0.smithi003.stdout: "close-test-pid93680": { 2024-09-22T03:58:09.928 INFO:tasks.workunit.client.0.smithi003.stdout: "start_epoch": 9, 2024-09-22T03:58:09.928 INFO:tasks.workunit.client.0.smithi003.stdout: "start_stamp": "2024-09-22T03:58:04.894866+0000", 2024-09-22T03:58:09.928 INFO:tasks.workunit.client.0.smithi003.stdout: "gid": 15650, 2024-09-22T03:58:09.928 INFO:tasks.workunit.client.0.smithi003.stdout: "addr": "172.21.15.3:0/474596603", 2024-09-22T03:58:09.928 INFO:tasks.workunit.client.0.smithi003.stdout: "metadata": { 2024-09-22T03:58:09.928 INFO:tasks.workunit.client.0.smithi003.stdout: "arch": "x86_64", 2024-09-22T03:58:09.928 INFO:tasks.workunit.client.0.smithi003.stdout: "ceph_release": "squid", 2024-09-22T03:58:09.928 INFO:tasks.workunit.client.0.smithi003.stdout: "ceph_version": "ceph version 19.3.0-5092-g62da54b4 (62da54b4b83a96ca9976b7e1d0c8272a564a2208) squid (dev)", 2024-09-22T03:58:09.928 INFO:tasks.workunit.client.0.smithi003.stdout: "ceph_version_short": "19.3.0-5092-g62da54b4", 2024-09-22T03:58:09.928 INFO:tasks.workunit.client.0.smithi003.stdout: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-09-22T03:58:09.928 INFO:tasks.workunit.client.0.smithi003.stdout: "distro": "centos", 2024-09-22T03:58:09.928 INFO:tasks.workunit.client.0.smithi003.stdout: "distro_description": "CentOS Stream 9", 2024-09-22T03:58:09.928 INFO:tasks.workunit.client.0.smithi003.stdout: "distro_version": "9", 2024-09-22T03:58:09.928 INFO:tasks.workunit.client.0.smithi003.stdout: "foo": "bar", 2024-09-22T03:58:09.928 INFO:tasks.workunit.client.0.smithi003.stdout: "hostname": "smithi003", 2024-09-22T03:58:09.929 INFO:tasks.workunit.client.0.smithi003.stdout: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024", 2024-09-22T03:58:09.929 INFO:tasks.workunit.client.0.smithi003.stdout: "kernel_version": "5.14.0-508.el9.x86_64", 2024-09-22T03:58:09.929 INFO:tasks.workunit.client.0.smithi003.stdout: "mem_swap_kb": "0", 2024-09-22T03:58:09.929 INFO:tasks.workunit.client.0.smithi003.stdout: "mem_total_kb": "32493488", 2024-09-22T03:58:09.929 INFO:tasks.workunit.client.0.smithi003.stdout: "os": "Linux", 2024-09-22T03:58:09.929 INFO:tasks.workunit.client.0.smithi003.stdout: "this": "that" 2024-09-22T03:58:09.929 INFO:tasks.workunit.client.0.smithi003.stdout: }, 2024-09-22T03:58:09.929 INFO:tasks.workunit.client.0.smithi003.stdout: "task_status": {} 2024-09-22T03:58:09.929 INFO:tasks.workunit.client.0.smithi003.stdout: } 2024-09-22T03:58:09.929 INFO:tasks.workunit.client.0.smithi003.stdout: } 2024-09-22T03:58:09.929 INFO:tasks.workunit.client.0.smithi003.stdout: } 2024-09-22T03:58:09.929 INFO:tasks.workunit.client.0.smithi003.stdout: } 2024-09-22T03:58:09.929 INFO:tasks.workunit.client.0.smithi003.stdout:} 2024-09-22T03:58:09.929 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:58:09.929 INFO:tasks.workunit.client.0.smithi003.stdout:attempt 1 of 20 2024-09-22T03:58:14.988 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosServicePP.Close (10124 ms) 2024-09-22T03:58:14.988 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 4 tests from LibRadosServicePP (35236 ms total) 2024-09-22T03:58:14.988 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:58:14.988 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:58:14.988 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 4 tests from 1 test suite ran. (35236 ms total) 2024-09-22T03:58:14.988 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 4 tests. 2024-09-22T03:58:14.991 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:58:14.991 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:58:14.991 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_c_write_operations 2024-09-22T03:58:15.014 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:58:15.014 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 8 tests from 2 test suites. 2024-09-22T03:58:15.014 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:58:15.014 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibradosCWriteOps 2024-09-22T03:58:15.014 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibradosCWriteOps.NewDelete 2024-09-22T03:58:15.014 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-09-22T03:58:15.014 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from LibradosCWriteOps (0 ms total) 2024-09-22T03:58:15.014 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:58:15.014 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 7 tests from LibRadosCWriteOps 2024-09-22T03:58:15.014 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosCWriteOps.assertExists 2024-09-22T03:58:17.333 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosCWriteOps.assertExists (2320 ms) 2024-09-22T03:58:17.333 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-09-22T03:58:20.346 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosCWriteOps.WriteOpAssertVersion (3013 ms) 2024-09-22T03:58:20.346 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosCWriteOps.Xattrs 2024-09-22T03:58:23.359 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosCWriteOps.Xattrs (3013 ms) 2024-09-22T03:58:23.359 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosCWriteOps.Write 2024-09-22T03:58:26.377 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosCWriteOps.Write (3018 ms) 2024-09-22T03:58:26.378 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosCWriteOps.Exec 2024-09-22T03:58:29.387 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosCWriteOps.Exec (3010 ms) 2024-09-22T03:58:29.388 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosCWriteOps.WriteSame 2024-09-22T03:58:32.406 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosCWriteOps.WriteSame (3018 ms) 2024-09-22T03:58:32.406 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] LibRadosCWriteOps.CmpExt 2024-09-22T03:58:47.330 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] LibRadosCWriteOps.CmpExt (14925 ms) 2024-09-22T03:58:47.330 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 7 tests from LibRadosCWriteOps (32317 ms total) 2024-09-22T03:58:47.331 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:58:47.331 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:58:47.331 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 8 tests from 2 test suites ran. (32317 ms total) 2024-09-22T03:58:47.331 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 8 tests. 2024-09-22T03:58:47.333 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:58:47.334 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:58:47.334 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_api_c_read_operations 2024-09-22T03:58:47.356 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:58:47.356 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 17 tests from 1 test suite. 2024-09-22T03:58:47.357 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:58:47.357 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 17 tests from CReadOpsTest 2024-09-22T03:58:49.337 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] CReadOpsTest.NewDelete 2024-09-22T03:58:49.337 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] CReadOpsTest.NewDelete (0 ms) 2024-09-22T03:58:49.337 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] CReadOpsTest.SetOpFlags 2024-09-22T03:58:49.862 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] CReadOpsTest.SetOpFlags (524 ms) 2024-09-22T03:58:49.862 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] CReadOpsTest.AssertExists 2024-09-22T03:58:49.919 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] CReadOpsTest.AssertExists (57 ms) 2024-09-22T03:58:49.919 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] CReadOpsTest.AssertVersion 2024-09-22T03:58:49.993 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] CReadOpsTest.AssertVersion (75 ms) 2024-09-22T03:58:49.993 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] CReadOpsTest.CmpXattr 2024-09-22T03:58:50.269 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] CReadOpsTest.CmpXattr (276 ms) 2024-09-22T03:58:50.269 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] CReadOpsTest.Read 2024-09-22T03:58:50.346 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] CReadOpsTest.Read (77 ms) 2024-09-22T03:58:50.346 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] CReadOpsTest.Checksum 2024-09-22T03:58:50.460 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] CReadOpsTest.Checksum (114 ms) 2024-09-22T03:58:50.460 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] CReadOpsTest.RWOrderedRead 2024-09-22T03:58:50.507 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] CReadOpsTest.RWOrderedRead (47 ms) 2024-09-22T03:58:50.507 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] CReadOpsTest.ShortRead 2024-09-22T03:58:50.576 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] CReadOpsTest.ShortRead (68 ms) 2024-09-22T03:58:50.576 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] CReadOpsTest.Exec 2024-09-22T03:58:50.632 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] CReadOpsTest.Exec (57 ms) 2024-09-22T03:58:50.632 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] CReadOpsTest.ExecUserBuf 2024-09-22T03:58:50.694 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] CReadOpsTest.ExecUserBuf (62 ms) 2024-09-22T03:58:50.694 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] CReadOpsTest.Stat 2024-09-22T03:58:50.747 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] CReadOpsTest.Stat (53 ms) 2024-09-22T03:58:50.747 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] CReadOpsTest.Stat2 2024-09-22T03:58:50.809 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] CReadOpsTest.Stat2 (62 ms) 2024-09-22T03:58:50.809 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] CReadOpsTest.Omap 2024-09-22T03:58:50.989 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] CReadOpsTest.Omap (179 ms) 2024-09-22T03:58:50.989 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] CReadOpsTest.OmapNuls 2024-09-22T03:58:51.144 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] CReadOpsTest.OmapNuls (156 ms) 2024-09-22T03:58:51.144 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] CReadOpsTest.GetXattrs 2024-09-22T03:58:51.272 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] CReadOpsTest.GetXattrs (128 ms) 2024-09-22T03:58:51.273 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] CReadOpsTest.CmpExt 2024-09-22T03:58:51.329 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] CReadOpsTest.CmpExt (57 ms) 2024-09-22T03:58:51.346 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 17 tests from CReadOpsTest (1992 ms total) 2024-09-22T03:58:51.346 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:58:51.346 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:58:51.346 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 17 tests from 1 test suite ran. (3989 ms total) 2024-09-22T03:58:51.346 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 17 tests. 2024-09-22T03:58:51.348 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:58:51.348 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:58:51.348 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_list_parallel 2024-09-22T03:58:55.064 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[93938]: starting. 2024-09-22T03:58:55.064 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[93938]: creating pool ceph_test_rados_list_parallel.smithi003-93937 2024-09-22T03:58:55.064 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[93938]: created object 0... 2024-09-22T03:58:55.064 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[93938]: created object 25... 2024-09-22T03:58:55.064 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[93938]: created object 49... 2024-09-22T03:58:55.064 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[93938]: finishing. 2024-09-22T03:58:55.064 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[93938]: shutting down. 2024-09-22T03:58:55.243 INFO:tasks.workunit.client.0.smithi003.stdout:process_2_[93939]: starting. 2024-09-22T03:58:55.243 INFO:tasks.workunit.client.0.smithi003.stdout:process_2_[93939]: listing objects. 2024-09-22T03:58:55.243 INFO:tasks.workunit.client.0.smithi003.stdout:process_2_[93939]: listed object 0... 2024-09-22T03:58:55.243 INFO:tasks.workunit.client.0.smithi003.stdout:process_2_[93939]: listed object 25... 2024-09-22T03:58:55.243 INFO:tasks.workunit.client.0.smithi003.stdout:process_2_[93939]: saw 50 objects 2024-09-22T03:58:55.243 INFO:tasks.workunit.client.0.smithi003.stdout:process_2_[93939]: shutting down. 2024-09-22T03:58:56.287 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:58:56.287 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[93974]: starting. 2024-09-22T03:58:56.287 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[93974]: creating pool ceph_test_rados_list_parallel.smithi003-93937 2024-09-22T03:58:56.287 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[93974]: created object 0... 2024-09-22T03:58:56.287 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[93974]: created object 25... 2024-09-22T03:58:56.287 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[93974]: created object 49... 2024-09-22T03:58:56.287 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[93974]: finishing. 2024-09-22T03:58:56.287 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[93974]: shutting down. 2024-09-22T03:58:56.810 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:58:56.810 INFO:tasks.workunit.client.0.smithi003.stdout:process_4_[93975]: starting. 2024-09-22T03:58:56.810 INFO:tasks.workunit.client.0.smithi003.stdout:process_4_[93975]: listing objects. 2024-09-22T03:58:56.810 INFO:tasks.workunit.client.0.smithi003.stdout:process_4_[93975]: listed object 0... 2024-09-22T03:58:56.810 INFO:tasks.workunit.client.0.smithi003.stdout:process_4_[93975]: listed object 25... 2024-09-22T03:58:56.810 INFO:tasks.workunit.client.0.smithi003.stdout:process_4_[93975]: saw 43 objects 2024-09-22T03:58:56.811 INFO:tasks.workunit.client.0.smithi003.stdout:process_4_[93975]: shutting down. 2024-09-22T03:58:57.278 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:58:57.279 INFO:tasks.workunit.client.0.smithi003.stdout:process_5_[93976]: starting. 2024-09-22T03:58:57.279 INFO:tasks.workunit.client.0.smithi003.stdout:process_5_[93976]: removed 25 objects... 2024-09-22T03:58:57.279 INFO:tasks.workunit.client.0.smithi003.stdout:process_5_[93976]: removed half of the objects 2024-09-22T03:58:57.279 INFO:tasks.workunit.client.0.smithi003.stdout:process_5_[93976]: removed 50 objects... 2024-09-22T03:58:57.279 INFO:tasks.workunit.client.0.smithi003.stdout:process_5_[93976]: removed 50 objects 2024-09-22T03:58:57.279 INFO:tasks.workunit.client.0.smithi003.stdout:process_5_[93976]: shutting down. 2024-09-22T03:58:58.428 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:58:58.428 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:58:58.428 INFO:tasks.workunit.client.0.smithi003.stdout:process_6_[94025]: starting. 2024-09-22T03:58:58.428 INFO:tasks.workunit.client.0.smithi003.stdout:process_6_[94025]: creating pool ceph_test_rados_list_parallel.smithi003-93937 2024-09-22T03:58:58.428 INFO:tasks.workunit.client.0.smithi003.stdout:process_6_[94025]: created object 0... 2024-09-22T03:58:58.428 INFO:tasks.workunit.client.0.smithi003.stdout:process_6_[94025]: created object 25... 2024-09-22T03:58:58.428 INFO:tasks.workunit.client.0.smithi003.stdout:process_6_[94025]: created object 49... 2024-09-22T03:58:58.429 INFO:tasks.workunit.client.0.smithi003.stdout:process_6_[94025]: finishing. 2024-09-22T03:58:58.429 INFO:tasks.workunit.client.0.smithi003.stdout:process_6_[94025]: shutting down. 2024-09-22T03:58:59.063 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:58:59.064 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:58:59.064 INFO:tasks.workunit.client.0.smithi003.stdout:process_7_[94026]: starting. 2024-09-22T03:58:59.064 INFO:tasks.workunit.client.0.smithi003.stdout:process_7_[94026]: listing objects. 2024-09-22T03:58:59.064 INFO:tasks.workunit.client.0.smithi003.stdout:process_7_[94026]: listed object 0... 2024-09-22T03:58:59.064 INFO:tasks.workunit.client.0.smithi003.stdout:process_7_[94026]: listed object 25... 2024-09-22T03:58:59.064 INFO:tasks.workunit.client.0.smithi003.stdout:process_7_[94026]: listed object 50... 2024-09-22T03:58:59.064 INFO:tasks.workunit.client.0.smithi003.stdout:process_7_[94026]: saw 53 objects 2024-09-22T03:58:59.064 INFO:tasks.workunit.client.0.smithi003.stdout:process_7_[94026]: shutting down. 2024-09-22T03:58:59.683 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:58:59.683 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:58:59.683 INFO:tasks.workunit.client.0.smithi003.stdout:process_8_[94027]: starting. 2024-09-22T03:58:59.683 INFO:tasks.workunit.client.0.smithi003.stdout:process_8_[94027]: added 25 objects... 2024-09-22T03:58:59.683 INFO:tasks.workunit.client.0.smithi003.stdout:process_8_[94027]: added half of the objects 2024-09-22T03:58:59.683 INFO:tasks.workunit.client.0.smithi003.stdout:process_8_[94027]: added 50 objects... 2024-09-22T03:58:59.683 INFO:tasks.workunit.client.0.smithi003.stdout:process_8_[94027]: added 50 objects 2024-09-22T03:58:59.683 INFO:tasks.workunit.client.0.smithi003.stdout:process_8_[94027]: shutting down. 2024-09-22T03:59:00.776 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:00.776 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:00.776 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:00.776 INFO:tasks.workunit.client.0.smithi003.stdout:process_9_[94076]: starting. 2024-09-22T03:59:00.776 INFO:tasks.workunit.client.0.smithi003.stdout:process_9_[94076]: creating pool ceph_test_rados_list_parallel.smithi003-93937 2024-09-22T03:59:00.777 INFO:tasks.workunit.client.0.smithi003.stdout:process_9_[94076]: created object 0... 2024-09-22T03:59:00.777 INFO:tasks.workunit.client.0.smithi003.stdout:process_9_[94076]: created object 25... 2024-09-22T03:59:00.777 INFO:tasks.workunit.client.0.smithi003.stdout:process_9_[94076]: created object 49... 2024-09-22T03:59:00.777 INFO:tasks.workunit.client.0.smithi003.stdout:process_9_[94076]: finishing. 2024-09-22T03:59:00.777 INFO:tasks.workunit.client.0.smithi003.stdout:process_9_[94076]: shutting down. 2024-09-22T03:59:01.513 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:01.513 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:01.513 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:01.514 INFO:tasks.workunit.client.0.smithi003.stdout:process_10_[94077]: starting. 2024-09-22T03:59:01.514 INFO:tasks.workunit.client.0.smithi003.stdout:process_10_[94077]: listing objects. 2024-09-22T03:59:01.514 INFO:tasks.workunit.client.0.smithi003.stdout:process_10_[94077]: listed object 0... 2024-09-22T03:59:01.514 INFO:tasks.workunit.client.0.smithi003.stdout:process_10_[94077]: listed object 25... 2024-09-22T03:59:01.514 INFO:tasks.workunit.client.0.smithi003.stdout:process_10_[94077]: listed object 50... 2024-09-22T03:59:01.514 INFO:tasks.workunit.client.0.smithi003.stdout:process_10_[94077]: listed object 75... 2024-09-22T03:59:01.514 INFO:tasks.workunit.client.0.smithi003.stdout:process_10_[94077]: saw 99 objects 2024-09-22T03:59:01.514 INFO:tasks.workunit.client.0.smithi003.stdout:process_10_[94077]: shutting down. 2024-09-22T03:59:02.293 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:02.293 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:02.293 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:02.294 INFO:tasks.workunit.client.0.smithi003.stdout:process_13_[94080]: starting. 2024-09-22T03:59:02.294 INFO:tasks.workunit.client.0.smithi003.stdout:process_13_[94080]: removed 25 objects... 2024-09-22T03:59:02.294 INFO:tasks.workunit.client.0.smithi003.stdout:process_13_[94080]: removed half of the objects 2024-09-22T03:59:02.294 INFO:tasks.workunit.client.0.smithi003.stdout:process_13_[94080]: removed 50 objects... 2024-09-22T03:59:02.294 INFO:tasks.workunit.client.0.smithi003.stdout:process_13_[94080]: removed 50 objects 2024-09-22T03:59:02.294 INFO:tasks.workunit.client.0.smithi003.stdout:process_13_[94080]: shutting down. 2024-09-22T03:59:02.415 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:02.415 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:02.415 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:02.415 INFO:tasks.workunit.client.0.smithi003.stdout:process_11_[94078]: starting. 2024-09-22T03:59:02.416 INFO:tasks.workunit.client.0.smithi003.stdout:process_11_[94078]: added 25 objects... 2024-09-22T03:59:02.416 INFO:tasks.workunit.client.0.smithi003.stdout:process_11_[94078]: added half of the objects 2024-09-22T03:59:02.416 INFO:tasks.workunit.client.0.smithi003.stdout:process_11_[94078]: added 50 objects... 2024-09-22T03:59:02.416 INFO:tasks.workunit.client.0.smithi003.stdout:process_11_[94078]: added 50 objects 2024-09-22T03:59:02.416 INFO:tasks.workunit.client.0.smithi003.stdout:process_11_[94078]: shutting down. 2024-09-22T03:59:02.494 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:02.494 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:02.494 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:02.494 INFO:tasks.workunit.client.0.smithi003.stdout:process_12_[94079]: starting. 2024-09-22T03:59:02.494 INFO:tasks.workunit.client.0.smithi003.stdout:process_12_[94079]: added 25 objects... 2024-09-22T03:59:02.494 INFO:tasks.workunit.client.0.smithi003.stdout:process_12_[94079]: added half of the objects 2024-09-22T03:59:02.494 INFO:tasks.workunit.client.0.smithi003.stdout:process_12_[94079]: added 50 objects... 2024-09-22T03:59:02.494 INFO:tasks.workunit.client.0.smithi003.stdout:process_12_[94079]: added 50 objects 2024-09-22T03:59:02.494 INFO:tasks.workunit.client.0.smithi003.stdout:process_12_[94079]: shutting down. 2024-09-22T03:59:03.680 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:03.680 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:03.680 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:03.680 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:03.680 INFO:tasks.workunit.client.0.smithi003.stdout:process_14_[94161]: starting. 2024-09-22T03:59:03.680 INFO:tasks.workunit.client.0.smithi003.stdout:process_14_[94161]: creating pool ceph_test_rados_list_parallel.smithi003-93937 2024-09-22T03:59:03.680 INFO:tasks.workunit.client.0.smithi003.stdout:process_14_[94161]: created object 0... 2024-09-22T03:59:03.680 INFO:tasks.workunit.client.0.smithi003.stdout:process_14_[94161]: created object 25... 2024-09-22T03:59:03.680 INFO:tasks.workunit.client.0.smithi003.stdout:process_14_[94161]: created object 49... 2024-09-22T03:59:03.680 INFO:tasks.workunit.client.0.smithi003.stdout:process_14_[94161]: finishing. 2024-09-22T03:59:03.681 INFO:tasks.workunit.client.0.smithi003.stdout:process_14_[94161]: shutting down. 2024-09-22T03:59:04.215 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:04.215 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:04.215 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:04.215 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:04.215 INFO:tasks.workunit.client.0.smithi003.stdout:process_15_[94162]: starting. 2024-09-22T03:59:04.215 INFO:tasks.workunit.client.0.smithi003.stdout:process_15_[94162]: listing objects. 2024-09-22T03:59:04.215 INFO:tasks.workunit.client.0.smithi003.stdout:process_15_[94162]: listed object 0... 2024-09-22T03:59:04.215 INFO:tasks.workunit.client.0.smithi003.stdout:process_15_[94162]: listed object 25... 2024-09-22T03:59:04.215 INFO:tasks.workunit.client.0.smithi003.stdout:process_15_[94162]: listed object 50... 2024-09-22T03:59:04.215 INFO:tasks.workunit.client.0.smithi003.stdout:process_15_[94162]: listed object 75... 2024-09-22T03:59:04.216 INFO:tasks.workunit.client.0.smithi003.stdout:process_15_[94162]: listed object 100... 2024-09-22T03:59:04.216 INFO:tasks.workunit.client.0.smithi003.stdout:process_15_[94162]: listed object 125... 2024-09-22T03:59:04.216 INFO:tasks.workunit.client.0.smithi003.stdout:process_15_[94162]: saw 150 objects 2024-09-22T03:59:04.216 INFO:tasks.workunit.client.0.smithi003.stdout:process_15_[94162]: shutting down. 2024-09-22T03:59:04.718 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:04.718 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:04.718 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:04.718 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:04.718 INFO:tasks.workunit.client.0.smithi003.stdout:process_16_[94163]: starting. 2024-09-22T03:59:04.718 INFO:tasks.workunit.client.0.smithi003.stdout:process_16_[94163]: added 25 objects... 2024-09-22T03:59:04.718 INFO:tasks.workunit.client.0.smithi003.stdout:process_16_[94163]: added half of the objects 2024-09-22T03:59:04.718 INFO:tasks.workunit.client.0.smithi003.stdout:process_16_[94163]: added 50 objects... 2024-09-22T03:59:04.719 INFO:tasks.workunit.client.0.smithi003.stdout:process_16_[94163]: added 50 objects 2024-09-22T03:59:04.719 INFO:tasks.workunit.client.0.smithi003.stdout:process_16_[94163]: shutting down. 2024-09-22T03:59:05.313 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:05.313 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:05.313 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:05.314 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:05.314 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:05.314 INFO:tasks.workunit.client.0.smithi003.stdout:******* SUCCESS ********** 2024-09-22T03:59:05.315 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:59:05.315 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:59:05.315 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_open_pools_parallel 2024-09-22T03:59:09.016 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[94229]: starting. 2024-09-22T03:59:09.016 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[94229]: creating pool ceph_test_rados_open_pools_parallel.smithi003-94228 2024-09-22T03:59:09.016 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[94229]: created object 0... 2024-09-22T03:59:09.016 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[94229]: created object 25... 2024-09-22T03:59:09.016 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[94229]: created object 49... 2024-09-22T03:59:09.016 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[94229]: finishing. 2024-09-22T03:59:09.016 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[94229]: shutting down. 2024-09-22T03:59:09.332 INFO:tasks.workunit.client.0.smithi003.stdout:process_2_[94230]: starting. 2024-09-22T03:59:09.332 INFO:tasks.workunit.client.0.smithi003.stdout:process_2_[94230]: rados_pool_create. 2024-09-22T03:59:09.332 INFO:tasks.workunit.client.0.smithi003.stdout:process_2_[94230]: rados_ioctx_create. 2024-09-22T03:59:09.332 INFO:tasks.workunit.client.0.smithi003.stdout:process_2_[94230]: shutting down. 2024-09-22T03:59:13.000 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:13.000 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[94265]: starting. 2024-09-22T03:59:13.000 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[94265]: creating pool ceph_test_rados_open_pools_parallel.smithi003-94228 2024-09-22T03:59:13.000 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[94265]: created object 0... 2024-09-22T03:59:13.000 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[94265]: created object 25... 2024-09-22T03:59:13.000 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[94265]: created object 49... 2024-09-22T03:59:13.000 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[94265]: finishing. 2024-09-22T03:59:13.000 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[94265]: shutting down. 2024-09-22T03:59:13.313 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:13.314 INFO:tasks.workunit.client.0.smithi003.stdout:process_4_[94266]: starting. 2024-09-22T03:59:13.314 INFO:tasks.workunit.client.0.smithi003.stdout:process_4_[94266]: rados_pool_create. 2024-09-22T03:59:13.314 INFO:tasks.workunit.client.0.smithi003.stdout:process_4_[94266]: rados_ioctx_create. 2024-09-22T03:59:13.314 INFO:tasks.workunit.client.0.smithi003.stdout:process_4_[94266]: shutting down. 2024-09-22T03:59:13.317 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:13.317 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:13.317 INFO:tasks.workunit.client.0.smithi003.stdout:******* SUCCESS ********** 2024-09-22T03:59:13.318 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-09-22T03:59:13.318 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:59:13.318 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_rados_delete_pools_parallel 2024-09-22T03:59:16.972 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[94302]: starting. 2024-09-22T03:59:16.972 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[94302]: creating pool ceph_test_rados_delete_pools_parallel.smithi003-94301 2024-09-22T03:59:16.973 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[94302]: created object 0... 2024-09-22T03:59:16.973 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[94302]: created object 25... 2024-09-22T03:59:16.973 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[94302]: created object 49... 2024-09-22T03:59:16.973 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[94302]: finishing. 2024-09-22T03:59:16.973 INFO:tasks.workunit.client.0.smithi003.stdout:process_1_[94302]: shutting down. 2024-09-22T03:59:17.366 INFO:tasks.workunit.client.0.smithi003.stdout:process_2_[94303]: starting. 2024-09-22T03:59:17.366 INFO:tasks.workunit.client.0.smithi003.stdout:process_2_[94303]: deleting pool ceph_test_rados_delete_pools_parallel.smithi003-94301 2024-09-22T03:59:17.366 INFO:tasks.workunit.client.0.smithi003.stdout:process_2_[94303]: shutting down. 2024-09-22T03:59:21.002 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:21.002 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[94338]: starting. 2024-09-22T03:59:21.002 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[94338]: creating pool ceph_test_rados_delete_pools_parallel.smithi003-94301 2024-09-22T03:59:21.003 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[94338]: created object 0... 2024-09-22T03:59:21.003 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[94338]: created object 25... 2024-09-22T03:59:21.003 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[94338]: created object 49... 2024-09-22T03:59:21.003 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[94338]: finishing. 2024-09-22T03:59:21.003 INFO:tasks.workunit.client.0.smithi003.stdout:process_3_[94338]: shutting down. 2024-09-22T03:59:21.187 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:21.188 INFO:tasks.workunit.client.0.smithi003.stdout:process_5_[94340]: starting. 2024-09-22T03:59:21.188 INFO:tasks.workunit.client.0.smithi003.stdout:process_5_[94340]: listing objects. 2024-09-22T03:59:21.188 INFO:tasks.workunit.client.0.smithi003.stdout:process_5_[94340]: listed object 0... 2024-09-22T03:59:21.188 INFO:tasks.workunit.client.0.smithi003.stdout:process_5_[94340]: listed object 25... 2024-09-22T03:59:21.188 INFO:tasks.workunit.client.0.smithi003.stdout:process_5_[94340]: saw 50 objects 2024-09-22T03:59:21.188 INFO:tasks.workunit.client.0.smithi003.stdout:process_5_[94340]: shutting down. 2024-09-22T03:59:21.396 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:21.397 INFO:tasks.workunit.client.0.smithi003.stdout:process_4_[94339]: starting. 2024-09-22T03:59:21.397 INFO:tasks.workunit.client.0.smithi003.stdout:process_4_[94339]: deleting pool ceph_test_rados_delete_pools_parallel.smithi003-94301 2024-09-22T03:59:21.397 INFO:tasks.workunit.client.0.smithi003.stdout:process_4_[94339]: shutting down. 2024-09-22T03:59:21.398 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:21.398 INFO:tasks.workunit.client.0.smithi003.stdout:******************************* 2024-09-22T03:59:21.398 INFO:tasks.workunit.client.0.smithi003.stdout:******* SUCCESS ********** 2024-09-22T03:59:21.399 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-22T03:59:21.400 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:59:21.400 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 1 -eq 1 ']' 2024-09-22T03:59:21.400 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' cls = ec_io ']' 2024-09-22T03:59:21.400 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' cls = ec_list ']' 2024-09-22T03:59:21.400 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_neorados_cls 2024-09-22T03:59:21.418 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:59:21.418 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 1 test from 1 test suite. 2024-09-22T03:59:21.418 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:59:21.418 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from NeoRadosCls 2024-09-22T03:59:21.418 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosCls.DNE 2024-09-22T03:59:24.413 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosCls.DNE (2994 ms) 2024-09-22T03:59:24.413 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from NeoRadosCls (2994 ms total) 2024-09-22T03:59:24.413 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:59:24.413 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:59:24.413 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 1 test from 1 test suite ran. (2994 ms total) 2024-09-22T03:59:24.413 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 1 test. 2024-09-22T03:59:24.415 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-22T03:59:24.415 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:59:24.415 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 1 -eq 1 ']' 2024-09-22T03:59:24.415 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' cmd = ec_io ']' 2024-09-22T03:59:24.415 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' cmd = ec_list ']' 2024-09-22T03:59:24.415 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_neorados_cmd 2024-09-22T03:59:24.435 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:59:24.436 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 3 tests from 1 test suite. 2024-09-22T03:59:24.436 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:59:24.436 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 3 tests from NeoRadosCmd 2024-09-22T03:59:24.436 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosCmd.MonDescribe 2024-09-22T03:59:26.422 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosCmd.MonDescribe (1987 ms) 2024-09-22T03:59:26.423 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosCmd.OSDCmd 2024-09-22T03:59:28.314 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosCmd.OSDCmd (1892 ms) 2024-09-22T03:59:28.314 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosCmd.PGCmd 2024-09-22T03:59:31.322 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosCmd.PGCmd (3008 ms) 2024-09-22T03:59:31.322 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 3 tests from NeoRadosCmd (6887 ms total) 2024-09-22T03:59:31.322 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:59:31.322 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:59:31.323 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 3 tests from 1 test suite ran. (6887 ms total) 2024-09-22T03:59:31.323 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 3 tests. 2024-09-22T03:59:31.325 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-22T03:59:31.325 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:59:31.325 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 1 -eq 1 ']' 2024-09-22T03:59:31.325 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' handler_error = ec_io ']' 2024-09-22T03:59:31.325 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' handler_error = ec_list ']' 2024-09-22T03:59:31.325 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_neorados_handler_error 2024-09-22T03:59:31.344 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:59:31.344 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 1 test from 1 test suite. 2024-09-22T03:59:31.344 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:59:31.344 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from neocls_handler_error 2024-09-22T03:59:31.344 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] neocls_handler_error.test_handler_error 2024-09-22T03:59:34.341 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] neocls_handler_error.test_handler_error (2997 ms) 2024-09-22T03:59:34.341 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 1 test from neocls_handler_error (2997 ms total) 2024-09-22T03:59:34.342 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T03:59:34.342 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T03:59:34.342 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 1 test from 1 test suite ran. (2997 ms total) 2024-09-22T03:59:34.342 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 1 test. 2024-09-22T03:59:34.344 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-22T03:59:34.344 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T03:59:34.344 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 1 -eq 1 ']' 2024-09-22T03:59:34.344 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' io = ec_io ']' 2024-09-22T03:59:34.344 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' io = ec_list ']' 2024-09-22T03:59:34.344 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_neorados_io 2024-09-22T03:59:34.365 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T03:59:34.365 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 14 tests from 1 test suite. 2024-09-22T03:59:34.365 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T03:59:34.365 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 14 tests from NeoRadosIo 2024-09-22T03:59:34.365 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosIo.Limits 2024-09-22T03:59:35.345 INFO:tasks.workunit.client.0.smithi003.stderr:Not supported by crimson yet. Skipped 2024-09-22T03:59:36.353 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosIo.Limits (1989 ms) 2024-09-22T03:59:36.353 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosIo.SimpleWrite 2024-09-22T03:59:39.365 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosIo.SimpleWrite (3012 ms) 2024-09-22T03:59:39.365 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosIo.ReadOp 2024-09-22T03:59:42.378 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosIo.ReadOp (3013 ms) 2024-09-22T03:59:42.378 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosIo.SparseRead 2024-09-22T03:59:45.388 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosIo.SparseRead (3010 ms) 2024-09-22T03:59:45.388 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosIo.RoundTrip 2024-09-22T03:59:48.321 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosIo.RoundTrip (2933 ms) 2024-09-22T03:59:48.321 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-09-22T03:59:51.332 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosIo.ReadIntoBuufferlist (3011 ms) 2024-09-22T03:59:51.332 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-09-22T03:59:54.347 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosIo.OverlappingWriteRoundTrip (3015 ms) 2024-09-22T03:59:54.347 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-09-22T03:59:57.359 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosIo.WriteFullRoundTrip (3012 ms) 2024-09-22T03:59:57.359 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosIo.AppendRoundTrip 2024-09-22T04:00:00.380 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosIo.AppendRoundTrip (3021 ms) 2024-09-22T04:00:00.380 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosIo.Trunc 2024-09-22T04:00:03.395 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosIo.Trunc (3014 ms) 2024-09-22T04:00:03.395 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosIo.Remove 2024-09-22T04:00:06.417 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosIo.Remove (3023 ms) 2024-09-22T04:00:06.417 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosIo.XattrsRoundTrip 2024-09-22T04:00:09.433 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosIo.XattrsRoundTrip (3016 ms) 2024-09-22T04:00:09.433 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosIo.RmXattr 2024-09-22T04:00:12.449 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosIo.RmXattr (3016 ms) 2024-09-22T04:00:12.449 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosIo.GetXattrs 2024-09-22T04:00:15.460 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosIo.GetXattrs (3011 ms) 2024-09-22T04:00:15.460 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 14 tests from NeoRadosIo (41096 ms total) 2024-09-22T04:00:15.460 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T04:00:15.461 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T04:00:15.461 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 14 tests from 1 test suite ran. (41096 ms total) 2024-09-22T04:00:15.461 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 14 tests. 2024-09-22T04:00:15.465 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-22T04:00:15.465 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T04:00:15.465 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 1 -eq 1 ']' 2024-09-22T04:00:15.466 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' ec_io = ec_io ']' 2024-09-22T04:00:15.466 INFO:tasks.workunit.client.0.smithi003.stderr:+ echo 'Skipping EC with Crimson' 2024-09-22T04:00:15.466 INFO:tasks.workunit.client.0.smithi003.stdout:Skipping EC with Crimson 2024-09-22T04:00:15.467 INFO:tasks.workunit.client.0.smithi003.stderr:+ continue 2024-09-22T04:00:15.467 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-22T04:00:15.467 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T04:00:15.467 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 1 -eq 1 ']' 2024-09-22T04:00:15.467 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' list = ec_io ']' 2024-09-22T04:00:15.467 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' list = ec_list ']' 2024-09-22T04:00:15.467 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_neorados_list 2024-09-22T04:00:15.487 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T04:00:15.487 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 3 tests from 1 test suite. 2024-09-22T04:00:15.487 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T04:00:15.487 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 3 tests from NeoradosList 2024-09-22T04:00:15.487 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoradosList.ListObjects 2024-09-22T04:00:18.326 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoradosList.ListObjects (2839 ms) 2024-09-22T04:00:18.326 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoradosList.ListObjectsNS 2024-09-22T04:00:22.345 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoradosList.ListObjectsNS (4019 ms) 2024-09-22T04:00:22.346 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoradosList.ListObjectsMany 2024-09-22T04:00:36.342 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoradosList.ListObjectsMany (13997 ms) 2024-09-22T04:00:36.342 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 3 tests from NeoradosList (20855 ms total) 2024-09-22T04:00:36.343 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T04:00:36.343 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T04:00:36.343 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 3 tests from 1 test suite ran. (20855 ms total) 2024-09-22T04:00:36.343 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 3 tests. 2024-09-22T04:00:36.346 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-22T04:00:36.346 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T04:00:36.346 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 1 -eq 1 ']' 2024-09-22T04:00:36.346 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' ec_list = ec_io ']' 2024-09-22T04:00:36.346 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' ec_list = ec_list ']' 2024-09-22T04:00:36.346 INFO:tasks.workunit.client.0.smithi003.stderr:+ echo 'Skipping EC with Crimson' 2024-09-22T04:00:36.346 INFO:tasks.workunit.client.0.smithi003.stdout:Skipping EC with Crimson 2024-09-22T04:00:36.347 INFO:tasks.workunit.client.0.smithi003.stderr:+ continue 2024-09-22T04:00:36.347 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-22T04:00:36.347 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T04:00:36.348 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 1 -eq 1 ']' 2024-09-22T04:00:36.348 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' misc = ec_io ']' 2024-09-22T04:00:36.348 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' misc = ec_list ']' 2024-09-22T04:00:36.348 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_neorados_misc 2024-09-22T04:00:36.369 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T04:00:36.369 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 12 tests from 1 test suite. 2024-09-22T04:00:36.369 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T04:00:36.369 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 12 tests from NeoRadosMisc 2024-09-22T04:00:36.369 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosMisc.Version 2024-09-22T04:00:38.347 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosMisc.Version (1978 ms) 2024-09-22T04:00:38.347 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosMisc.WaitOSDMap 2024-09-22T04:00:40.357 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosMisc.WaitOSDMap (2010 ms) 2024-09-22T04:00:40.357 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosMisc.LongName 2024-09-22T04:00:43.331 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosMisc.LongName (2974 ms) 2024-09-22T04:00:43.332 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosMisc.LongLocator 2024-09-22T04:00:44.338 INFO:tasks.workunit.client.0.smithi003.stderr:Not supported by crimson yet. Skipped 2024-09-22T04:00:45.341 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosMisc.LongLocator (2009 ms) 2024-09-22T04:00:45.341 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosMisc.LongNamespace 2024-09-22T04:00:48.357 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosMisc.LongNamespace (3017 ms) 2024-09-22T04:00:48.357 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosMisc.LongAttrName 2024-09-22T04:00:51.368 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosMisc.LongAttrName (3010 ms) 2024-09-22T04:00:51.368 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosMisc.Exec 2024-09-22T04:00:54.377 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosMisc.Exec (3010 ms) 2024-09-22T04:00:54.378 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosMisc.Operate1 2024-09-22T04:00:57.394 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosMisc.Operate1 (3017 ms) 2024-09-22T04:00:57.394 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosMisc.Operate2 2024-09-22T04:01:00.418 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosMisc.Operate2 (3024 ms) 2024-09-22T04:01:00.418 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosMisc.BigObject 2024-09-22T04:01:03.339 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosMisc.BigObject (2921 ms) 2024-09-22T04:01:03.339 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosMisc.BigAttr 2024-09-22T04:01:05.349 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosMisc.BigAttr (2010 ms) 2024-09-22T04:01:05.349 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosMisc.WriteSame 2024-09-22T04:01:06.359 INFO:tasks.workunit.client.0.smithi003.stderr:Not supported by crimson yet. Skipped 2024-09-22T04:01:07.362 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosMisc.WriteSame (2013 ms) 2024-09-22T04:01:07.362 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 12 tests from NeoRadosMisc (30993 ms total) 2024-09-22T04:01:07.362 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T04:01:07.363 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T04:01:07.363 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 12 tests from 1 test suite ran. (30993 ms total) 2024-09-22T04:01:07.363 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 12 tests. 2024-09-22T04:01:07.366 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-22T04:01:07.367 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T04:01:07.367 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 1 -eq 1 ']' 2024-09-22T04:01:07.367 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' pool = ec_io ']' 2024-09-22T04:01:07.367 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' pool = ec_list ']' 2024-09-22T04:01:07.367 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_neorados_pool 2024-09-22T04:01:07.390 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T04:01:07.390 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 6 tests from 1 test suite. 2024-09-22T04:01:07.390 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T04:01:07.390 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 6 tests from NeoRadosPools 2024-09-22T04:01:07.390 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosPools.PoolList 2024-09-22T04:01:09.380 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosPools.PoolList (1990 ms) 2024-09-22T04:01:09.381 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosPools.PoolLookup 2024-09-22T04:01:11.394 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosPools.PoolLookup (2013 ms) 2024-09-22T04:01:11.394 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-09-22T04:01:13.340 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosPools.PoolLookupOtherInstance (1947 ms) 2024-09-22T04:01:13.340 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosPools.PoolDelete 2024-09-22T04:01:17.351 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosPools.PoolDelete (4010 ms) 2024-09-22T04:01:17.351 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosPools.PoolCreateDelete 2024-09-22T04:01:19.355 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosPools.PoolCreateDelete (2005 ms) 2024-09-22T04:01:19.355 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-09-22T04:01:21.365 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosPools.PoolCreateWithCrushRule (2010 ms) 2024-09-22T04:01:21.365 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 6 tests from NeoRadosPools (13975 ms total) 2024-09-22T04:01:21.365 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T04:01:21.365 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T04:01:21.365 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 6 tests from 1 test suite ran. (13975 ms total) 2024-09-22T04:01:21.365 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 6 tests. 2024-09-22T04:01:21.369 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-22T04:01:21.369 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T04:01:21.369 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 1 -eq 1 ']' 2024-09-22T04:01:21.369 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' read_operations = ec_io ']' 2024-09-22T04:01:21.369 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' read_operations = ec_list ']' 2024-09-22T04:01:21.369 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_neorados_read_operations 2024-09-22T04:01:21.393 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T04:01:21.393 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 14 tests from 1 test suite. 2024-09-22T04:01:21.393 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T04:01:21.393 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 14 tests from NeoRadosReadOps 2024-09-22T04:01:21.393 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosReadOps.SetOpFlags 2024-09-22T04:01:24.379 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosReadOps.SetOpFlags (2987 ms) 2024-09-22T04:01:24.379 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosReadOps.AssertExists 2024-09-22T04:01:27.398 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosReadOps.AssertExists (3019 ms) 2024-09-22T04:01:27.398 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosReadOps.AssertVersion 2024-09-22T04:01:30.409 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosReadOps.AssertVersion (3011 ms) 2024-09-22T04:01:30.410 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosReadOps.CmpXattr 2024-09-22T04:01:33.346 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosReadOps.CmpXattr (2936 ms) 2024-09-22T04:01:33.346 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosReadOps.Read 2024-09-22T04:01:36.360 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosReadOps.Read (3015 ms) 2024-09-22T04:01:36.360 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosReadOps.Checksum 2024-09-22T04:01:39.376 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosReadOps.Checksum (3015 ms) 2024-09-22T04:01:39.376 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosReadOps.RWOrderedRead 2024-09-22T04:01:42.398 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosReadOps.RWOrderedRead (3023 ms) 2024-09-22T04:01:42.398 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosReadOps.ShortRead 2024-09-22T04:01:45.400 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosReadOps.ShortRead (3002 ms) 2024-09-22T04:01:45.400 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosReadOps.Exec 2024-09-22T04:01:48.419 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosReadOps.Exec (3019 ms) 2024-09-22T04:01:48.419 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosReadOps.Stat 2024-09-22T04:01:51.437 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosReadOps.Stat (3018 ms) 2024-09-22T04:01:51.437 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosReadOps.Omap 2024-09-22T04:01:54.447 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosReadOps.Omap (3009 ms) 2024-09-22T04:01:54.447 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosReadOps.OmapNuls 2024-09-22T04:01:57.458 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosReadOps.OmapNuls (3012 ms) 2024-09-22T04:01:57.459 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosReadOps.GetXattrs 2024-09-22T04:02:00.480 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosReadOps.GetXattrs (3022 ms) 2024-09-22T04:02:00.480 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosReadOps.CmpExt 2024-09-22T04:02:03.490 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosReadOps.CmpExt (3010 ms) 2024-09-22T04:02:03.491 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 14 tests from NeoRadosReadOps (42098 ms total) 2024-09-22T04:02:03.491 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T04:02:03.491 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T04:02:03.491 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 14 tests from 1 test suite ran. (42098 ms total) 2024-09-22T04:02:03.491 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 14 tests. 2024-09-22T04:02:03.495 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-22T04:02:03.495 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T04:02:03.495 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 1 -eq 1 ']' 2024-09-22T04:02:03.495 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' snapshots = ec_io ']' 2024-09-22T04:02:03.495 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' snapshots = ec_list ']' 2024-09-22T04:02:03.495 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_neorados_snapshots 2024-09-22T04:02:03.514 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T04:02:03.514 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 11 tests from 2 test suites. 2024-09-22T04:02:03.514 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T04:02:03.515 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 5 tests from NeoRadosSnapshots 2024-09-22T04:02:03.515 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosSnapshots.SnapList 2024-09-22T04:02:08.520 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosSnapshots.SnapList (5006 ms) 2024-09-22T04:02:08.520 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosSnapshots.SnapRemove 2024-09-22T04:02:13.358 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosSnapshots.SnapRemove (4838 ms) 2024-09-22T04:02:13.358 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosSnapshots.Rollback 2024-09-22T04:02:17.376 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosSnapshots.Rollback (4018 ms) 2024-09-22T04:02:17.376 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosSnapshots.SnapGetName 2024-09-22T04:02:22.399 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosSnapshots.SnapGetName (5023 ms) 2024-09-22T04:02:22.399 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-09-22T04:02:29.430 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosSnapshots.SnapCreateRemove (7031 ms) 2024-09-22T04:02:29.431 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 5 tests from NeoRadosSnapshots (25916 ms total) 2024-09-22T04:02:29.431 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T04:02:29.431 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 6 tests from NeoRadosSelfManagedSnaps 2024-09-22T04:02:29.431 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-09-22T04:02:34.453 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosSelfManagedSnaps.Snap (5022 ms) 2024-09-22T04:02:34.453 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-09-22T04:02:35.455 INFO:tasks.workunit.client.0.smithi003.stderr:Not supported by crimson yet. Skipped 2024-09-22T04:02:36.461 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosSelfManagedSnaps.Rollback (2009 ms) 2024-09-22T04:02:36.461 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-09-22T04:02:37.464 INFO:tasks.workunit.client.0.smithi003.stderr:Not supported by crimson yet. Skipped 2024-09-22T04:02:38.363 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (1902 ms) 2024-09-22T04:02:38.364 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-09-22T04:02:44.373 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosSelfManagedSnaps.Bug11677 (6010 ms) 2024-09-22T04:02:44.373 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-09-22T04:02:48.411 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4037 ms) 2024-09-22T04:02:48.411 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-09-22T04:02:51.429 INFO:tasks.workunit.client.0.smithi003.stdout:Deleting snap 3 in pool ReusePurgedSnapsmithi003-95252-11. 2024-09-22T04:02:52.435 INFO:tasks.workunit.client.0.smithi003.stdout:Waiting for snaps to purge. 2024-09-22T04:03:08.376 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (19965 ms) 2024-09-22T04:03:08.376 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 6 tests from NeoRadosSelfManagedSnaps (38945 ms total) 2024-09-22T04:03:08.376 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T04:03:08.376 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T04:03:08.376 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 11 tests from 2 test suites ran. (64861 ms total) 2024-09-22T04:03:08.376 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 11 tests. 2024-09-22T04:03:08.380 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-22T04:03:08.380 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T04:03:08.380 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 1 -eq 1 ']' 2024-09-22T04:03:08.380 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' watch_notify = ec_io ']' 2024-09-22T04:03:08.381 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' watch_notify = ec_list ']' 2024-09-22T04:03:08.381 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_neorados_watch_notify 2024-09-22T04:03:08.404 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T04:03:08.404 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 2 tests from 1 test suite. 2024-09-22T04:03:08.404 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T04:03:08.404 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 2 tests from NeoRadosWatchNotify 2024-09-22T04:03:08.404 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosWatchNotify.WatchNotify 2024-09-22T04:03:10.761 INFO:tasks.workunit.client.0.smithi003.stdout:handle_notify cookie 31384384 notify_id 2890512990216 notifier_gid 15958 2024-09-22T04:03:11.385 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosWatchNotify.WatchNotify (2981 ms) 2024-09-22T04:03:11.385 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-09-22T04:03:13.895 INFO:tasks.workunit.client.0.smithi003.stdout:Trying... 2024-09-22T04:03:13.899 INFO:tasks.workunit.client.0.smithi003.stdout:handle_notify cookie 43328992 notify_id 2903397892098 notifier_gid 15961 2024-09-22T04:03:13.899 INFO:tasks.workunit.client.0.smithi003.stdout:Waiting for 3.000000000s 2024-09-22T04:03:14.898 INFO:tasks.workunit.client.0.smithi003.stdout:Timed out. 2024-09-22T04:03:14.925 INFO:tasks.workunit.client.0.smithi003.stdout:Flushing... 2024-09-22T04:03:14.925 INFO:tasks.workunit.client.0.smithi003.stdout:Flushed... 2024-09-22T04:03:18.373 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (6987 ms) 2024-09-22T04:03:18.373 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 2 tests from NeoRadosWatchNotify (9968 ms total) 2024-09-22T04:03:18.373 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T04:03:18.373 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T04:03:18.373 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 2 tests from 1 test suite ran. (9969 ms total) 2024-09-22T04:03:18.373 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 2 tests. 2024-09-22T04:03:18.375 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-09-22T04:03:18.376 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T04:03:18.376 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 1 -eq 1 ']' 2024-09-22T04:03:18.376 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' write_operations = ec_io ']' 2024-09-22T04:03:18.376 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' write_operations = ec_list ']' 2024-09-22T04:03:18.376 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph_test_neorados_write_operations 2024-09-22T04:03:18.399 INFO:tasks.workunit.client.0.smithi003.stdout:Running main() from gmock_main.cc 2024-09-22T04:03:18.399 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] Running 7 tests from 1 test suite. 2024-09-22T04:03:18.399 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment set-up. 2024-09-22T04:03:18.399 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 7 tests from NeoRadosWriteOps 2024-09-22T04:03:18.399 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosWriteOps.AssertExists 2024-09-22T04:03:21.385 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosWriteOps.AssertExists (2986 ms) 2024-09-22T04:03:21.386 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosWriteOps.AssertVersion 2024-09-22T04:03:24.397 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosWriteOps.AssertVersion (3012 ms) 2024-09-22T04:03:24.398 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosWriteOps.Xattrs 2024-09-22T04:03:27.414 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosWriteOps.Xattrs (3017 ms) 2024-09-22T04:03:27.414 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosWriteOps.Write 2024-09-22T04:03:30.431 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosWriteOps.Write (3017 ms) 2024-09-22T04:03:30.432 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosWriteOps.Exec 2024-09-22T04:03:33.444 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosWriteOps.Exec (3012 ms) 2024-09-22T04:03:33.444 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosWriteOps.WriteSame 2024-09-22T04:03:36.459 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosWriteOps.WriteSame (3015 ms) 2024-09-22T04:03:36.459 INFO:tasks.workunit.client.0.smithi003.stdout:[ RUN ] NeoRadosWriteOps.CmpExt 2024-09-22T04:03:50.852 INFO:tasks.workunit.client.0.smithi003.stdout:[ OK ] NeoRadosWriteOps.CmpExt (14394 ms) 2024-09-22T04:03:50.852 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] 7 tests from NeoRadosWriteOps (32453 ms total) 2024-09-22T04:03:50.853 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-22T04:03:50.853 INFO:tasks.workunit.client.0.smithi003.stdout:[----------] Global test environment tear-down 2024-09-22T04:03:50.853 INFO:tasks.workunit.client.0.smithi003.stdout:[==========] 7 tests from 1 test suite ran. (32453 ms total) 2024-09-22T04:03:50.853 INFO:tasks.workunit.client.0.smithi003.stdout:[ PASSED ] 7 tests. 2024-09-22T04:03:50.855 INFO:tasks.workunit.client.0.smithi003.stderr:+ ret=0 2024-09-22T04:03:50.855 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -eq 1 ']' 2024-09-22T04:03:50.855 INFO:tasks.workunit.client.0.smithi003.stderr:+ exit 0 2024-09-22T04:03:50.855 INFO:tasks.workunit.client.0.smithi003.stderr:+ cleanup 2024-09-22T04:03:50.856 INFO:tasks.workunit.client.0.smithi003.stderr:+ pkill -P 29639 2024-09-22T04:03:50.874 INFO:tasks.workunit.client.0.smithi003.stderr:+ true 2024-09-22T04:03:50.875 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-22T04:03:50.875 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-22T04:03:50.960 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-09-22T04:03:50.962 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-09-22T04:03:50.962 DEBUG:teuthology.orchestra.run.smithi003:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=62da54b4b83a96ca9976b7e1d0c8272a564a2208 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-09-22T04:03:50.989 INFO:tasks.workunit.client.0.smithi003.stderr:++ uuidgen 2024-09-22T04:03:50.999 INFO:tasks.workunit.client.0.smithi003.stderr:+ p=7c1c124a-aa8e-411d-9731-1f1389755458 2024-09-22T04:03:50.999 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool create 7c1c124a-aa8e-411d-9731-1f1389755458 12 2024-09-22T04:03:51.110 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.094+0000 7f8c21688640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:51.112 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.111+0000 7f8c21688640 1 Processor -- start 2024-09-22T04:03:51.112 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.111+0000 7f8c21688640 1 -- start start 2024-09-22T04:03:51.114 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.113+0000 7f8c21688640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c1c136050 0x7f8c1c136460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:51.114 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.113+0000 7f8c21688640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c1c135c90 con 0x7f8c1c1369a0 2024-09-22T04:03:51.114 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.113+0000 7f8c21688640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8c1c0a15a0 con 0x7f8c1c136050 2024-09-22T04:03:51.114 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.113+0000 7f8c1affd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8c1c136050 0x7f8c1c136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:51.115 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.113+0000 7f8c127fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8c1c1369a0 0x7f8c1c132790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44334/0 (socket says 172.21.15.3:44334) 2024-09-22T04:03:51.115 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.113+0000 7f8c127fc640 1 -- 172.21.15.3:0/2587997991 learned_addr learned my addr 172.21.15.3:0/2587997991 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T04:03:51.115 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.114+0000 7f8c1affd640 1 -- 172.21.15.3:0/2587997991 >> v1:172.21.15.3:6789/0 conn(0x7f8c1c1369a0 legacy=0x7f8c1c132790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:51.116 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.114+0000 7f8c1affd640 1 -- 172.21.15.3:0/2587997991 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1c0a0d70 con 0x7f8c1c136050 2024-09-22T04:03:51.116 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.114+0000 7f8c1a7fc640 1 -- 172.21.15.3:0/2587997991 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2749358274 0 0) 0x7f8c1c135c90 con 0x7f8c1c1369a0 2024-09-22T04:03:51.116 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.114+0000 7f8c1affd640 1 --2- 172.21.15.3:0/2587997991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c1c136050 0x7f8c1c136460 crc :-1 s=READY pgs=12861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91a8d76f813840c server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:51.116 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.114+0000 7f8c1a7fc640 1 -- 172.21.15.3:0/2587997991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8c0c00e040 con 0x7f8c1c136050 2024-09-22T04:03:51.116 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.114+0000 7f8c1a7fc640 1 -- 172.21.15.3:0/2587997991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8c0c007670 con 0x7f8c1c136050 2024-09-22T04:03:51.116 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.114+0000 7f8c1a7fc640 1 -- 172.21.15.3:0/2587997991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8c0c007910 con 0x7f8c1c136050 2024-09-22T04:03:51.117 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.115+0000 7f8c21688640 1 -- 172.21.15.3:0/2587997991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c1c136050 msgr2=0x7f8c1c136460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:51.117 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.115+0000 7f8c21688640 1 --2- 172.21.15.3:0/2587997991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c1c136050 0x7f8c1c136460 crc :-1 s=READY pgs=12861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:51.117 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.115+0000 7f8c21688640 1 -- 172.21.15.3:0/2587997991 shutdown_connections 2024-09-22T04:03:51.117 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.115+0000 7f8c21688640 1 --2- 172.21.15.3:0/2587997991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c1c136050 0x7f8c1c136460 unknown :-1 s=CLOSED pgs=12861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:51.117 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.115+0000 7f8c21688640 1 -- 172.21.15.3:0/2587997991 >> 172.21.15.3:0/2587997991 conn(0x7f8c1c0a6200 msgr2=0x7f8c1c0a6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:51.117 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.115+0000 7f8c21688640 1 -- 172.21.15.3:0/2587997991 shutdown_connections 2024-09-22T04:03:51.117 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.115+0000 7f8c21688640 1 -- 172.21.15.3:0/2587997991 wait complete. 2024-09-22T04:03:51.117 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.116+0000 7f8c21688640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:51.119 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.118+0000 7f8c21688640 1 Processor -- start 2024-09-22T04:03:51.120 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.119+0000 7f8c21688640 1 -- start start 2024-09-22T04:03:51.122 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.121+0000 7f8c21688640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c1c136050 0x7f8c1c1ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:51.122 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.121+0000 7f8c21688640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c133270 con 0x7f8c1c136050 2024-09-22T04:03:51.123 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.121+0000 7f8c1affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c1c136050 0x7f8c1c1ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:51.123 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.121+0000 7f8c1affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c1c136050 0x7f8c1c1ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33068/0 (socket says 172.21.15.3:33068) 2024-09-22T04:03:51.123 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.121+0000 7f8c1affd640 1 -- 172.21.15.3:0/3369483844 learned_addr learned my addr 172.21.15.3:0/3369483844 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:51.123 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.122+0000 7f8c1affd640 1 -- 172.21.15.3:0/3369483844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1c12f420 con 0x7f8c1c136050 2024-09-22T04:03:51.124 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.122+0000 7f8c1affd640 1 --2- 172.21.15.3:0/3369483844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c1c136050 0x7f8c1c1ca1d0 crc :-1 s=READY pgs=12862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:51.124 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.122+0000 7f8c13fff640 1 -- 172.21.15.3:0/3369483844 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8c0c00e040 con 0x7f8c1c136050 2024-09-22T04:03:51.124 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.122+0000 7f8c21688640 1 -- 172.21.15.3:0/3369483844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1c0a4610 con 0x7f8c1c136050 2024-09-22T04:03:51.124 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.122+0000 7f8c13fff640 1 -- 172.21.15.3:0/3369483844 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8c0c012ab0 con 0x7f8c1c136050 2024-09-22T04:03:51.124 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.122+0000 7f8c13fff640 1 -- 172.21.15.3:0/3369483844 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8c0c012d50 con 0x7f8c1c136050 2024-09-22T04:03:51.124 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.122+0000 7f8c21688640 1 -- 172.21.15.3:0/3369483844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1c132d90 con 0x7f8c1c136050 2024-09-22T04:03:51.126 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.125+0000 7f8c21688640 1 -- 172.21.15.3:0/3369483844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c1c0a7f70 con 0x7f8c1c136050 2024-09-22T04:03:51.126 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.125+0000 7f8c13fff640 1 -- 172.21.15.3:0/3369483844 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f8c0c0153f0 con 0x7f8c1c136050 2024-09-22T04:03:51.126 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.125+0000 7f8c13fff640 1 --2- 172.21.15.3:0/3369483844 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8bfc03ea80 0x7f8bfc040f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:51.126 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.125+0000 7f8c13fff640 1 -- 172.21.15.3:0/3369483844 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(702..702 src has 590..702) ==== 3431+0+0 (crc 0 0 0) 0x7f8c0c0516a0 con 0x7f8c1c136050 2024-09-22T04:03:51.127 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.126+0000 7f8c127fc640 1 --2- 172.21.15.3:0/3369483844 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8bfc03ea80 0x7f8bfc040f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:51.127 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.126+0000 7f8c127fc640 1 --2- 172.21.15.3:0/3369483844 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8bfc03ea80 0x7f8bfc040f90 crc :-1 s=READY pgs=3830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:51.132 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.131+0000 7f8c13fff640 1 -- 172.21.15.3:0/3369483844 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f8c0c00e5c0 con 0x7f8c1c136050 2024-09-22T04:03:51.332 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.330+0000 7f8c21688640 1 -- 172.21.15.3:0/3369483844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "pg_num": 12} v 0) -- 0x7f8c1c050fc0 con 0x7f8c1c136050 2024-09-22T04:03:51.855 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:51.854+0000 7f8c13fff640 1 -- 172.21.15.3:0/3369483844 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "pg_num": 12}]=0 pool '7c1c124a-aa8e-411d-9731-1f1389755458' created v703) ==== 176+0+0 (crc 0 0 0) 0x7f8c0c00e1a0 con 0x7f8c1c136050 2024-09-22T04:03:52.007 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.006+0000 7f8c21688640 1 -- 172.21.15.3:0/3369483844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "pg_num": 12} v 0) -- 0x7f8c1c131510 con 0x7f8c1c136050 2024-09-22T04:03:52.007 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.006+0000 7f8c13fff640 1 -- 172.21.15.3:0/3369483844 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "pg_num": 12}]=0 pool '7c1c124a-aa8e-411d-9731-1f1389755458' already exists v703) ==== 183+0+0 (crc 0 0 0) 0x7f8c0c00e1a0 con 0x7f8c1c136050 2024-09-22T04:03:52.007 INFO:tasks.workunit.client.0.smithi003.stderr:pool '7c1c124a-aa8e-411d-9731-1f1389755458' already exists 2024-09-22T04:03:52.010 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.008+0000 7f8c21688640 1 -- 172.21.15.3:0/3369483844 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8bfc03ea80 msgr2=0x7f8bfc040f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:52.010 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.008+0000 7f8c21688640 1 --2- 172.21.15.3:0/3369483844 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8bfc03ea80 0x7f8bfc040f90 crc :-1 s=READY pgs=3830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:52.010 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.008+0000 7f8c21688640 1 -- 172.21.15.3:0/3369483844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c1c136050 msgr2=0x7f8c1c1ca1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:52.010 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.008+0000 7f8c21688640 1 --2- 172.21.15.3:0/3369483844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c1c136050 0x7f8c1c1ca1d0 crc :-1 s=READY pgs=12862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:52.010 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.008+0000 7f8c21688640 1 -- 172.21.15.3:0/3369483844 shutdown_connections 2024-09-22T04:03:52.010 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.008+0000 7f8c21688640 1 --2- 172.21.15.3:0/3369483844 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8bfc03ea80 0x7f8bfc040f90 unknown :-1 s=CLOSED pgs=3830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:52.010 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.008+0000 7f8c21688640 1 --2- 172.21.15.3:0/3369483844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c1c136050 0x7f8c1c1ca1d0 unknown :-1 s=CLOSED pgs=12862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:52.010 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.008+0000 7f8c21688640 1 -- 172.21.15.3:0/3369483844 >> 172.21.15.3:0/3369483844 conn(0x7f8c1c0a6200 msgr2=0x7f8c1c1346f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:52.011 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.009+0000 7f8c21688640 1 -- 172.21.15.3:0/3369483844 shutdown_connections 2024-09-22T04:03:52.011 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.009+0000 7f8c21688640 1 -- 172.21.15.3:0/3369483844 wait complete. 2024-09-22T04:03:52.019 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool set-quota 7c1c124a-aa8e-411d-9731-1f1389755458 max_objects 10 2024-09-22T04:03:52.116 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.106+0000 7f4e9eb91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:52.117 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.116+0000 7f4e9eb91640 1 Processor -- start 2024-09-22T04:03:52.117 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.116+0000 7f4e9eb91640 1 -- start start 2024-09-22T04:03:52.119 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.117+0000 7f4e9eb91640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4e981364c0 0x7f4e981368d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:52.119 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.117+0000 7f4e9eb91640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e980a16a0 con 0x7f4e98136e10 2024-09-22T04:03:52.120 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.117+0000 7f4e9eb91640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4e980a0a00 con 0x7f4e981364c0 2024-09-22T04:03:52.120 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.118+0000 7f4e9c92d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4e981364c0 0x7f4e981368d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:52.120 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.118+0000 7f4e9c92d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4e981364c0 0x7f4e981368d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33082/0 (socket says 172.21.15.3:33082) 2024-09-22T04:03:52.120 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.118+0000 7f4e9c92d640 1 -- 172.21.15.3:0/108085806 learned_addr learned my addr 172.21.15.3:0/108085806 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:52.120 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.118+0000 7f4e97fff640 1 --1- 172.21.15.3:0/108085806 >> v1:172.21.15.3:6789/0 conn(0x7f4e98136e10 0x7f4e9809f870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44342/0 (socket says 172.21.15.3:44342) 2024-09-22T04:03:52.120 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.118+0000 7f4e9c92d640 1 -- 172.21.15.3:0/108085806 >> v1:172.21.15.3:6789/0 conn(0x7f4e98136e10 legacy=0x7f4e9809f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:52.120 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.118+0000 7f4e9c92d640 1 -- 172.21.15.3:0/108085806 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e980a13b0 con 0x7f4e981364c0 2024-09-22T04:03:52.120 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.118+0000 7f4e977fe640 1 -- 172.21.15.3:0/108085806 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1319149505 0 0) 0x7f4e980a16a0 con 0x7f4e98136e10 2024-09-22T04:03:52.120 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.118+0000 7f4e9c92d640 1 --2- 172.21.15.3:0/108085806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e981364c0 0x7f4e981368d0 crc :-1 s=READY pgs=12864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae2550056efa720 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:52.120 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.118+0000 7f4e977fe640 1 -- 172.21.15.3:0/108085806 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4e8c009e50 con 0x7f4e981364c0 2024-09-22T04:03:52.120 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.118+0000 7f4e977fe640 1 -- 172.21.15.3:0/108085806 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4e8c00e230 con 0x7f4e981364c0 2024-09-22T04:03:52.120 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.118+0000 7f4e977fe640 1 -- 172.21.15.3:0/108085806 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4e8c00e4d0 con 0x7f4e981364c0 2024-09-22T04:03:52.121 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.119+0000 7f4e9eb91640 1 -- 172.21.15.3:0/108085806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e981364c0 msgr2=0x7f4e981368d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:52.121 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.119+0000 7f4e9eb91640 1 --2- 172.21.15.3:0/108085806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e981364c0 0x7f4e981368d0 crc :-1 s=READY pgs=12864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:52.121 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.119+0000 7f4e9eb91640 1 -- 172.21.15.3:0/108085806 shutdown_connections 2024-09-22T04:03:52.121 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.119+0000 7f4e9eb91640 1 --2- 172.21.15.3:0/108085806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e981364c0 0x7f4e981368d0 unknown :-1 s=CLOSED pgs=12864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:52.122 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.119+0000 7f4e9eb91640 1 -- 172.21.15.3:0/108085806 >> 172.21.15.3:0/108085806 conn(0x7f4e980a6270 msgr2=0x7f4e980a6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:52.122 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.119+0000 7f4e9eb91640 1 -- 172.21.15.3:0/108085806 shutdown_connections 2024-09-22T04:03:52.122 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.119+0000 7f4e9eb91640 1 -- 172.21.15.3:0/108085806 wait complete. 2024-09-22T04:03:52.122 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.119+0000 7f4e9eb91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:52.122 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.121+0000 7f4e9eb91640 1 Processor -- start 2024-09-22T04:03:52.122 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.121+0000 7f4e9eb91640 1 -- start start 2024-09-22T04:03:52.125 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.123+0000 7f4e9eb91640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e981364c0 0x7f4e981ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:52.125 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.123+0000 7f4e9eb91640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4e98136100 con 0x7f4e981364c0 2024-09-22T04:03:52.125 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.123+0000 7f4e9c92d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e981364c0 0x7f4e981ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:52.125 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.123+0000 7f4e9c92d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e981364c0 0x7f4e981ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33084/0 (socket says 172.21.15.3:33084) 2024-09-22T04:03:52.125 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.123+0000 7f4e9c92d640 1 -- 172.21.15.3:0/1623775122 learned_addr learned my addr 172.21.15.3:0/1623775122 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:52.125 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.123+0000 7f4e9c92d640 1 -- 172.21.15.3:0/1623775122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e980a4210 con 0x7f4e981364c0 2024-09-22T04:03:52.125 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.123+0000 7f4e9c92d640 1 --2- 172.21.15.3:0/1623775122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e981364c0 0x7f4e981ca850 crc :-1 s=READY pgs=12865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:52.125 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.123+0000 7f4e957fa640 1 -- 172.21.15.3:0/1623775122 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4e8c00edc0 con 0x7f4e981364c0 2024-09-22T04:03:52.125 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.124+0000 7f4e957fa640 1 -- 172.21.15.3:0/1623775122 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4e8c015420 con 0x7f4e981364c0 2024-09-22T04:03:52.125 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.124+0000 7f4e957fa640 1 -- 172.21.15.3:0/1623775122 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4e8c0156c0 con 0x7f4e981364c0 2024-09-22T04:03:52.125 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.124+0000 7f4e9eb91640 1 -- 172.21.15.3:0/1623775122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e98132260 con 0x7f4e981364c0 2024-09-22T04:03:52.125 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.124+0000 7f4e9eb91640 1 -- 172.21.15.3:0/1623775122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e981343c0 con 0x7f4e981364c0 2024-09-22T04:03:52.126 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.124+0000 7f4e9eb91640 1 -- 172.21.15.3:0/1623775122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e980a2230 con 0x7f4e981364c0 2024-09-22T04:03:52.126 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.125+0000 7f4e957fa640 1 -- 172.21.15.3:0/1623775122 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f4e8c015870 con 0x7f4e981364c0 2024-09-22T04:03:52.126 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.125+0000 7f4e957fa640 1 --2- 172.21.15.3:0/1623775122 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4e7c03ead0 0x7f4e7c040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:52.126 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.125+0000 7f4e957fa640 1 -- 172.21.15.3:0/1623775122 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(703..703 src has 590..703) ==== 3806+0+0 (crc 0 0 0) 0x7f4e8c051250 con 0x7f4e981364c0 2024-09-22T04:03:52.126 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.125+0000 7f4e97fff640 1 --2- 172.21.15.3:0/1623775122 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4e7c03ead0 0x7f4e7c040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:52.127 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.126+0000 7f4e97fff640 1 --2- 172.21.15.3:0/1623775122 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4e7c03ead0 0x7f4e7c040fe0 crc :-1 s=READY pgs=3831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:52.131 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.130+0000 7f4e957fa640 1 -- 172.21.15.3:0/1623775122 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f4e8c010a00 con 0x7f4e981364c0 2024-09-22T04:03:52.321 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.319+0000 7f4e9eb91640 1 -- 172.21.15.3:0/1623775122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_objects", "val": "10"} v 0) -- 0x7f4e98050fc0 con 0x7f4e981364c0 2024-09-22T04:03:52.863 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:52.862+0000 7f4e957fa640 1 -- 172.21.15.3:0/1623775122 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 v704) ==== 223+0+0 (crc 0 0 0) 0x7f4e8c011dd0 con 0x7f4e981364c0 2024-09-22T04:03:53.016 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.015+0000 7f4e9eb91640 1 -- 172.21.15.3:0/1623775122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_objects", "val": "10"} v 0) -- 0x7f4e98136de0 con 0x7f4e981364c0 2024-09-22T04:03:53.377 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.376+0000 7f4e957fa640 1 -- 172.21.15.3:0/1623775122 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 v705) ==== 223+0+0 (crc 0 0 0) 0x7f4e8c010310 con 0x7f4e981364c0 2024-09-22T04:03:53.377 INFO:tasks.workunit.client.0.smithi003.stderr:set-quota max_objects = 10 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 2024-09-22T04:03:53.380 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.378+0000 7f4e9eb91640 1 -- 172.21.15.3:0/1623775122 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4e7c03ead0 msgr2=0x7f4e7c040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:53.380 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.378+0000 7f4e9eb91640 1 --2- 172.21.15.3:0/1623775122 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4e7c03ead0 0x7f4e7c040fe0 crc :-1 s=READY pgs=3831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:53.380 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.378+0000 7f4e9eb91640 1 -- 172.21.15.3:0/1623775122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e981364c0 msgr2=0x7f4e981ca850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:53.380 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.378+0000 7f4e9eb91640 1 --2- 172.21.15.3:0/1623775122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e981364c0 0x7f4e981ca850 crc :-1 s=READY pgs=12865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:53.380 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.378+0000 7f4e9eb91640 1 -- 172.21.15.3:0/1623775122 shutdown_connections 2024-09-22T04:03:53.380 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.378+0000 7f4e9eb91640 1 --2- 172.21.15.3:0/1623775122 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4e7c03ead0 0x7f4e7c040fe0 unknown :-1 s=CLOSED pgs=3831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:53.380 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.378+0000 7f4e9eb91640 1 --2- 172.21.15.3:0/1623775122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e981364c0 0x7f4e981ca850 unknown :-1 s=CLOSED pgs=12865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:53.380 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.379+0000 7f4e9eb91640 1 -- 172.21.15.3:0/1623775122 >> 172.21.15.3:0/1623775122 conn(0x7f4e980a6270 msgr2=0x7f4e980714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:53.380 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.379+0000 7f4e9eb91640 1 -- 172.21.15.3:0/1623775122 shutdown_connections 2024-09-22T04:03:53.380 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.379+0000 7f4e9eb91640 1 -- 172.21.15.3:0/1623775122 wait complete. 2024-09-22T04:03:53.391 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool application enable 7c1c124a-aa8e-411d-9731-1f1389755458 rados 2024-09-22T04:03:53.498 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.487+0000 7f8f31591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:53.499 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.498+0000 7f8f31591640 1 Processor -- start 2024-09-22T04:03:53.499 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.498+0000 7f8f31591640 1 -- start start 2024-09-22T04:03:53.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.499+0000 7f8f31591640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8f2c09a420 0x7f8f2c09a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:53.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.500+0000 7f8f31591640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f2c09cdf0 con 0x7f8f2c1320b0 2024-09-22T04:03:53.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.500+0000 7f8f31591640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8f2c09e230 con 0x7f8f2c09a420 2024-09-22T04:03:53.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.500+0000 7f8f2affd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8f2c09a420 0x7f8f2c09a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:53.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.500+0000 7f8f2affd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8f2c09a420 0x7f8f2c09a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33086/0 (socket says 172.21.15.3:33086) 2024-09-22T04:03:53.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.500+0000 7f8f2a7fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8f2c1320b0 0x7f8f2c1324c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44352/0 (socket says 172.21.15.3:44352) 2024-09-22T04:03:53.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.500+0000 7f8f2affd640 1 -- 172.21.15.3:0/435590139 learned_addr learned my addr 172.21.15.3:0/435590139 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:53.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.500+0000 7f8f2affd640 1 -- 172.21.15.3:0/435590139 >> v1:172.21.15.3:6789/0 conn(0x7f8f2c1320b0 legacy=0x7f8f2c1324c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:53.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.500+0000 7f8f29ffb640 1 -- 172.21.15.3:0/435590139 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4003726190 0 0) 0x7f8f2c09cdf0 con 0x7f8f2c1320b0 2024-09-22T04:03:53.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.500+0000 7f8f2affd640 1 -- 172.21.15.3:0/435590139 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2c074690 con 0x7f8f2c09a420 2024-09-22T04:03:53.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.500+0000 7f8f2affd640 1 --2- 172.21.15.3:0/435590139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2c09a420 0x7f8f2c09a830 crc :-1 s=READY pgs=12867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d160384e42a3d973 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:53.503 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.500+0000 7f8f29ffb640 1 -- 172.21.15.3:0/435590139 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8f1c00ade0 con 0x7f8f2c09a420 2024-09-22T04:03:53.503 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.500+0000 7f8f29ffb640 1 -- 172.21.15.3:0/435590139 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8f1c00f340 con 0x7f8f2c09a420 2024-09-22T04:03:53.503 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.500+0000 7f8f29ffb640 1 -- 172.21.15.3:0/435590139 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8f1c00f5e0 con 0x7f8f2c09a420 2024-09-22T04:03:53.503 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.501+0000 7f8f31591640 1 -- 172.21.15.3:0/435590139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2c09a420 msgr2=0x7f8f2c09a830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:53.504 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.501+0000 7f8f31591640 1 --2- 172.21.15.3:0/435590139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2c09a420 0x7f8f2c09a830 crc :-1 s=READY pgs=12867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:53.504 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.502+0000 7f8f31591640 1 -- 172.21.15.3:0/435590139 shutdown_connections 2024-09-22T04:03:53.504 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.502+0000 7f8f31591640 1 --2- 172.21.15.3:0/435590139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2c09a420 0x7f8f2c09a830 unknown :-1 s=CLOSED pgs=12867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:53.504 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.502+0000 7f8f31591640 1 -- 172.21.15.3:0/435590139 >> 172.21.15.3:0/435590139 conn(0x7f8f2c0a0f60 msgr2=0x7f8f2c0a1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:53.504 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.502+0000 7f8f31591640 1 -- 172.21.15.3:0/435590139 shutdown_connections 2024-09-22T04:03:53.504 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.502+0000 7f8f31591640 1 -- 172.21.15.3:0/435590139 wait complete. 2024-09-22T04:03:53.504 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.502+0000 7f8f31591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:53.505 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.504+0000 7f8f31591640 1 Processor -- start 2024-09-22T04:03:53.505 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.504+0000 7f8f31591640 1 -- start start 2024-09-22T04:03:53.507 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.505+0000 7f8f31591640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2c09a420 0x7f8f2c1c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:53.507 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.505+0000 7f8f31591640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c08ebf0 con 0x7f8f2c09a420 2024-09-22T04:03:53.507 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.505+0000 7f8f2affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2c09a420 0x7f8f2c1c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:53.507 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.505+0000 7f8f2affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2c09a420 0x7f8f2c1c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33090/0 (socket says 172.21.15.3:33090) 2024-09-22T04:03:53.507 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.505+0000 7f8f2affd640 1 -- 172.21.15.3:0/1124479152 learned_addr learned my addr 172.21.15.3:0/1124479152 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:53.507 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.506+0000 7f8f2affd640 1 -- 172.21.15.3:0/1124479152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2c131510 con 0x7f8f2c09a420 2024-09-22T04:03:53.507 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.506+0000 7f8f2affd640 1 --2- 172.21.15.3:0/1124479152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2c09a420 0x7f8f2c1c8680 crc :-1 s=READY pgs=12868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:53.508 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.506+0000 7f8f237fe640 1 -- 172.21.15.3:0/1124479152 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8f1c011420 con 0x7f8f2c09a420 2024-09-22T04:03:53.508 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.506+0000 7f8f237fe640 1 -- 172.21.15.3:0/1124479152 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8f1c011580 con 0x7f8f2c09a420 2024-09-22T04:03:53.508 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.506+0000 7f8f31591640 1 -- 172.21.15.3:0/1124479152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2c0a3740 con 0x7f8f2c09a420 2024-09-22T04:03:53.508 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.506+0000 7f8f237fe640 1 -- 172.21.15.3:0/1124479152 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8f1c011820 con 0x7f8f2c09a420 2024-09-22T04:03:53.508 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.506+0000 7f8f31591640 1 -- 172.21.15.3:0/1124479152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f2c08e6c0 con 0x7f8f2c09a420 2024-09-22T04:03:53.509 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.507+0000 7f8f31591640 1 -- 172.21.15.3:0/1124479152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f2c06ee20 con 0x7f8f2c09a420 2024-09-22T04:03:53.509 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.508+0000 7f8f237fe640 1 -- 172.21.15.3:0/1124479152 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f8f1c0119d0 con 0x7f8f2c09a420 2024-09-22T04:03:53.509 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.508+0000 7f8f237fe640 1 --2- 172.21.15.3:0/1124479152 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8f0c03eb00 0x7f8f0c041010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:53.509 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.508+0000 7f8f237fe640 1 -- 172.21.15.3:0/1124479152 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(705..705 src has 590..705) ==== 3806+0+0 (crc 0 0 0) 0x7f8f1c050cf0 con 0x7f8f2c09a420 2024-09-22T04:03:53.510 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.508+0000 7f8f2a7fc640 1 --2- 172.21.15.3:0/1124479152 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8f0c03eb00 0x7f8f0c041010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:53.510 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.509+0000 7f8f2a7fc640 1 --2- 172.21.15.3:0/1124479152 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8f0c03eb00 0x7f8f0c041010 crc :-1 s=READY pgs=3832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:53.515 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.513+0000 7f8f237fe640 1 -- 172.21.15.3:0/1124479152 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f8f1c0852b0 con 0x7f8f2c09a420 2024-09-22T04:03:53.706 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:53.705+0000 7f8f31591640 1 -- 172.21.15.3:0/1124479152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "app": "rados"} v 0) -- 0x7f8f2c06eaa0 con 0x7f8f2c09a420 2024-09-22T04:03:54.381 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:54.380+0000 7f8f237fe640 1 -- 172.21.15.3:0/1124479152 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "app": "rados"}]=0 enabled application 'rados' on pool '7c1c124a-aa8e-411d-9731-1f1389755458' v706) ==== 213+0+0 (crc 0 0 0) 0x7f8f1c012af0 con 0x7f8f2c09a420 2024-09-22T04:03:54.530 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:54.529+0000 7f8f31591640 1 -- 172.21.15.3:0/1124479152 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "app": "rados"} v 0) -- 0x7f8f2c0749a0 con 0x7f8f2c09a420 2024-09-22T04:03:55.383 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.382+0000 7f8f237fe640 1 -- 172.21.15.3:0/1124479152 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "app": "rados"}]=0 enabled application 'rados' on pool '7c1c124a-aa8e-411d-9731-1f1389755458' v707) ==== 213+0+0 (crc 0 0 0) 0x7f8f1c00c760 con 0x7f8f2c09a420 2024-09-22T04:03:55.383 INFO:tasks.workunit.client.0.smithi003.stderr:enabled application 'rados' on pool '7c1c124a-aa8e-411d-9731-1f1389755458' 2024-09-22T04:03:55.388 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.386+0000 7f8f31591640 1 -- 172.21.15.3:0/1124479152 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8f0c03eb00 msgr2=0x7f8f0c041010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.388 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.386+0000 7f8f31591640 1 --2- 172.21.15.3:0/1124479152 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8f0c03eb00 0x7f8f0c041010 crc :-1 s=READY pgs=3832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.388 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.386+0000 7f8f31591640 1 -- 172.21.15.3:0/1124479152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2c09a420 msgr2=0x7f8f2c1c8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.388 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.386+0000 7f8f31591640 1 --2- 172.21.15.3:0/1124479152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2c09a420 0x7f8f2c1c8680 crc :-1 s=READY pgs=12868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.388 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.386+0000 7f8f31591640 1 -- 172.21.15.3:0/1124479152 shutdown_connections 2024-09-22T04:03:55.388 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.386+0000 7f8f31591640 1 --2- 172.21.15.3:0/1124479152 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8f0c03eb00 0x7f8f0c041010 crc :-1 s=CLOSED pgs=3832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.388 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.386+0000 7f8f31591640 1 --2- 172.21.15.3:0/1124479152 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f2c09a420 0x7f8f2c1c8680 unknown :-1 s=CLOSED pgs=12868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.388 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.386+0000 7f8f31591640 1 -- 172.21.15.3:0/1124479152 >> 172.21.15.3:0/1124479152 conn(0x7f8f2c0a0f60 msgr2=0x7f8f2c0710d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:55.388 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.386+0000 7f8f31591640 1 -- 172.21.15.3:0/1124479152 shutdown_connections 2024-09-22T04:03:55.388 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.386+0000 7f8f31591640 1 -- 172.21.15.3:0/1124479152 wait complete. 2024-09-22T04:03:55.404 INFO:tasks.workunit.client.0.smithi003.stderr:++ seq 1 10 2024-09-22T04:03:55.415 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:03:55.415 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p 7c1c124a-aa8e-411d-9731-1f1389755458 put obj1 /etc/passwd 2024-09-22T04:03:55.464 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.463+0000 7fbf03032ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.470 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.469+0000 7fbf03032ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.470 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.469+0000 7fbf03032ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.471 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.470+0000 7fbf03032ec0 1 Processor -- start 2024-09-22T04:03:55.471 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.470+0000 7fbf03032ec0 1 -- start start 2024-09-22T04:03:55.473 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.471+0000 7fbf03032ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x233e740 0x233f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.473 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.471+0000 7fbf03032ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x2213520 con 0x233e740 2024-09-22T04:03:55.473 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.471+0000 7fbefb19b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x233e740 0x233f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.473 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.471+0000 7fbefb19b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x233e740 0x233f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33100/0 (socket says 172.21.15.3:33100) 2024-09-22T04:03:55.473 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.471+0000 7fbefb19b640 1 -- 172.21.15.3:0/312185493 learned_addr learned my addr 172.21.15.3:0/312185493 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:55.473 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.472+0000 7fbefb19b640 1 -- 172.21.15.3:0/312185493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x21afe10 con 0x233e740 2024-09-22T04:03:55.473 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.472+0000 7fbefb19b640 1 --2- 172.21.15.3:0/312185493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x233e740 0x233f0f0 crc :-1 s=READY pgs=12871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9fb69eb6a198bb server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.473 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.472+0000 7fbef9998640 1 -- 172.21.15.3:0/312185493 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbee400d500 con 0x233e740 2024-09-22T04:03:55.473 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.472+0000 7fbef9998640 1 -- 172.21.15.3:0/312185493 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbee400d660 con 0x233e740 2024-09-22T04:03:55.474 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.472+0000 7fbef9998640 1 -- 172.21.15.3:0/312185493 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbee400d900 con 0x233e740 2024-09-22T04:03:55.474 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.472+0000 7fbf03032ec0 1 -- 172.21.15.3:0/312185493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x233e740 msgr2=0x233f0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.475 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.472+0000 7fbf03032ec0 1 --2- 172.21.15.3:0/312185493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x233e740 0x233f0f0 crc :-1 s=READY pgs=12871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.475 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.473+0000 7fbf03032ec0 1 -- 172.21.15.3:0/312185493 shutdown_connections 2024-09-22T04:03:55.475 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.473+0000 7fbf03032ec0 1 --2- 172.21.15.3:0/312185493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x233e740 0x233f0f0 unknown :-1 s=CLOSED pgs=12871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.475 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.473+0000 7fbf03032ec0 1 -- 172.21.15.3:0/312185493 >> 172.21.15.3:0/312185493 conn(0x21d3770 msgr2=0x23c7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:55.475 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.473+0000 7fbf03032ec0 1 -- 172.21.15.3:0/312185493 shutdown_connections 2024-09-22T04:03:55.475 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.473+0000 7fbf03032ec0 1 -- 172.21.15.3:0/312185493 wait complete. 2024-09-22T04:03:55.475 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.474+0000 7fbf03032ec0 1 Processor -- start 2024-09-22T04:03:55.475 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.474+0000 7fbf03032ec0 1 -- start start 2024-09-22T04:03:55.477 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.475+0000 7fbf03032ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x233e740 0x23d6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.477 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.475+0000 7fbf03032ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x2211280 con 0x233e740 2024-09-22T04:03:55.477 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.475+0000 7fbefb19b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x233e740 0x23d6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.477 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.475+0000 7fbefb19b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x233e740 0x23d6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33112/0 (socket says 172.21.15.3:33112) 2024-09-22T04:03:55.477 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.475+0000 7fbefb19b640 1 -- 172.21.15.3:0/3149987149 learned_addr learned my addr 172.21.15.3:0/3149987149 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:55.477 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.476+0000 7fbefb19b640 1 -- 172.21.15.3:0/3149987149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x21a1920 con 0x233e740 2024-09-22T04:03:55.477 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.476+0000 7fbefb19b640 1 --2- 172.21.15.3:0/3149987149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x233e740 0x23d6ba0 crc :-1 s=READY pgs=12872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.477 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.476+0000 7fbef8996640 1 -- 172.21.15.3:0/3149987149 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbee400fbf0 con 0x233e740 2024-09-22T04:03:55.477 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.476+0000 7fbef8996640 1 -- 172.21.15.3:0/3149987149 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbee400fd50 con 0x233e740 2024-09-22T04:03:55.478 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.476+0000 7fbf03032ec0 1 -- 172.21.15.3:0/3149987149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x21a1380 con 0x233e740 2024-09-22T04:03:55.478 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.476+0000 7fbef8996640 1 -- 172.21.15.3:0/3149987149 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbee400c360 con 0x233e740 2024-09-22T04:03:55.478 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.476+0000 7fbf03032ec0 1 -- 172.21.15.3:0/3149987149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x22acf00 con 0x233e740 2024-09-22T04:03:55.479 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.477+0000 7fbef8996640 1 -- 172.21.15.3:0/3149987149 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fbee400c510 con 0x233e740 2024-09-22T04:03:55.479 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.477+0000 7fbef8996640 1 --2- 172.21.15.3:0/3149987149 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbee003d360 0x7fbee003f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.479 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.477+0000 7fbef8996640 1 -- 172.21.15.3:0/3149987149 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(707..707 src has 590..707) ==== 3819+0+0 (crc 0 0 0) 0x7fbee4050470 con 0x233e740 2024-09-22T04:03:55.479 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.477+0000 7fbf03032ec0 1 --2- 172.21.15.3:0/3149987149 >> v2:172.21.15.3:6801/2889521405 conn(0x233eb50 0x233ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.479 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.477+0000 7fbf03032ec0 1 -- 172.21.15.3:0/3149987149 --> v2:172.21.15.3:6801/2889521405 -- osd_op(unknown.0.0:1 199.7 199:ff7b1f36:::obj1:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e707) -- 0x232cc30 con 0x233eb50 2024-09-22T04:03:55.479 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.478+0000 7fbefa99a640 1 --2- 172.21.15.3:0/3149987149 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbee003d360 0x7fbee003f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.480 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.478+0000 7fbefa99a640 1 --2- 172.21.15.3:0/3149987149 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbee003d360 0x7fbee003f870 crc :-1 s=READY pgs=3833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.480 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.478+0000 7fbefb99c640 1 --2- 172.21.15.3:0/3149987149 >> v2:172.21.15.3:6801/2889521405 conn(0x233eb50 0x233ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:03:55.483 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.482+0000 7fbefb99c640 1 --2- 172.21.15.3:0/3149987149 >> v2:172.21.15.3:6801/2889521405 conn(0x233eb50 0x233ce80 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.510 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.508+0000 7fbefb99c640 1 -- 172.21.15.3:0/3149987149 <== osd.0 v2:172.21.15.3:6801/2889521405 1 ==== osd_op_reply(1 obj1 [writefull 0~12875] v707'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbeec00a430 con 0x233eb50 2024-09-22T04:03:55.510 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.508+0000 7fbf03032ec0 1 -- 172.21.15.3:0/3149987149 >> v2:172.21.15.3:6801/2889521405 conn(0x233eb50 msgr2=0x233ce80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.510 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.508+0000 7fbf03032ec0 1 --2- 172.21.15.3:0/3149987149 >> v2:172.21.15.3:6801/2889521405 conn(0x233eb50 0x233ce80 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.510 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.508+0000 7fbf03032ec0 1 -- 172.21.15.3:0/3149987149 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbee003d360 msgr2=0x7fbee003f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.510 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.508+0000 7fbf03032ec0 1 --2- 172.21.15.3:0/3149987149 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbee003d360 0x7fbee003f870 crc :-1 s=READY pgs=3833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.510 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.508+0000 7fbf03032ec0 1 -- 172.21.15.3:0/3149987149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x233e740 msgr2=0x23d6ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.508+0000 7fbf03032ec0 1 --2- 172.21.15.3:0/3149987149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x233e740 0x23d6ba0 crc :-1 s=READY pgs=12872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.509+0000 7fbf03032ec0 1 -- 172.21.15.3:0/3149987149 shutdown_connections 2024-09-22T04:03:55.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.509+0000 7fbf03032ec0 1 --2- 172.21.15.3:0/3149987149 >> v2:172.21.15.3:6801/2889521405 conn(0x233eb50 0x233ce80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.509+0000 7fbf03032ec0 1 --2- 172.21.15.3:0/3149987149 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbee003d360 0x7fbee003f870 unknown :-1 s=CLOSED pgs=3833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.509+0000 7fbf03032ec0 1 --2- 172.21.15.3:0/3149987149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x233e740 0x23d6ba0 unknown :-1 s=CLOSED pgs=12872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.509+0000 7fbf03032ec0 1 -- 172.21.15.3:0/3149987149 >> 172.21.15.3:0/3149987149 conn(0x21d3770 msgr2=0x23c7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:55.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.509+0000 7fbf03032ec0 1 -- 172.21.15.3:0/3149987149 shutdown_connections 2024-09-22T04:03:55.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.509+0000 7fbf03032ec0 1 -- 172.21.15.3:0/3149987149 wait complete. 2024-09-22T04:03:55.512 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:03:55.512 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p 7c1c124a-aa8e-411d-9731-1f1389755458 put obj2 /etc/passwd 2024-09-22T04:03:55.551 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.550+0000 7fa52b722ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.556 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.555+0000 7fa52b722ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.557 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.555+0000 7fa52b722ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.557 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.556+0000 7fa52b722ec0 1 Processor -- start 2024-09-22T04:03:55.557 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.556+0000 7fa52b722ec0 1 -- start start 2024-09-22T04:03:55.559 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.558+0000 7fa52b722ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18ca150 0x19e4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.560 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.558+0000 7fa52b722ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x182e520 con 0x18ca150 2024-09-22T04:03:55.560 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.558+0000 7fa51effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18ca150 0x19e4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.560 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.558+0000 7fa51effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18ca150 0x19e4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33130/0 (socket says 172.21.15.3:33130) 2024-09-22T04:03:55.560 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.558+0000 7fa51effd640 1 -- 172.21.15.3:0/1590767450 learned_addr learned my addr 172.21.15.3:0/1590767450 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:55.560 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.558+0000 7fa51effd640 1 -- 172.21.15.3:0/1590767450 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x17cae10 con 0x18ca150 2024-09-22T04:03:55.560 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.558+0000 7fa51effd640 1 --2- 172.21.15.3:0/1590767450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18ca150 0x19e4950 crc :-1 s=READY pgs=12875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ec946de6599720 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.560 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.558+0000 7fa51dffb640 1 -- 172.21.15.3:0/1590767450 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa51000c5e0 con 0x18ca150 2024-09-22T04:03:55.560 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.559+0000 7fa51dffb640 1 -- 172.21.15.3:0/1590767450 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa51000c740 con 0x18ca150 2024-09-22T04:03:55.560 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.559+0000 7fa51dffb640 1 -- 172.21.15.3:0/1590767450 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa51000c9e0 con 0x18ca150 2024-09-22T04:03:55.561 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.559+0000 7fa52b722ec0 1 -- 172.21.15.3:0/1590767450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18ca150 msgr2=0x19e4950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.561 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.559+0000 7fa52b722ec0 1 --2- 172.21.15.3:0/1590767450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18ca150 0x19e4950 crc :-1 s=READY pgs=12875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.561 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.559+0000 7fa52b722ec0 1 -- 172.21.15.3:0/1590767450 shutdown_connections 2024-09-22T04:03:55.561 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.559+0000 7fa52b722ec0 1 --2- 172.21.15.3:0/1590767450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18ca150 0x19e4950 unknown :-1 s=CLOSED pgs=12875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.561 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.559+0000 7fa52b722ec0 1 -- 172.21.15.3:0/1590767450 >> 172.21.15.3:0/1590767450 conn(0x17ee770 msgr2=0x18c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:55.561 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.559+0000 7fa52b722ec0 1 -- 172.21.15.3:0/1590767450 shutdown_connections 2024-09-22T04:03:55.561 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.559+0000 7fa52b722ec0 1 -- 172.21.15.3:0/1590767450 wait complete. 2024-09-22T04:03:55.562 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.560+0000 7fa52b722ec0 1 Processor -- start 2024-09-22T04:03:55.562 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.560+0000 7fa52b722ec0 1 -- start start 2024-09-22T04:03:55.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.562+0000 7fa52b722ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18cacd0 0x18c23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.562+0000 7fa52b722ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x182c280 con 0x18cacd0 2024-09-22T04:03:55.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.562+0000 7fa51effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18cacd0 0x18c23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.562+0000 7fa51effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18cacd0 0x18c23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33132/0 (socket says 172.21.15.3:33132) 2024-09-22T04:03:55.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.562+0000 7fa51effd640 1 -- 172.21.15.3:0/2230308214 learned_addr learned my addr 172.21.15.3:0/2230308214 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:55.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.562+0000 7fa51effd640 1 -- 172.21.15.3:0/2230308214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x17bc920 con 0x18cacd0 2024-09-22T04:03:55.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.562+0000 7fa51effd640 1 --2- 172.21.15.3:0/2230308214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18cacd0 0x18c23e0 crc :-1 s=READY pgs=12876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.562+0000 7fa51cff9640 1 -- 172.21.15.3:0/2230308214 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa510013970 con 0x18cacd0 2024-09-22T04:03:55.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.562+0000 7fa51cff9640 1 -- 172.21.15.3:0/2230308214 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa510013ad0 con 0x18cacd0 2024-09-22T04:03:55.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.563+0000 7fa52b722ec0 1 -- 172.21.15.3:0/2230308214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x17bc380 con 0x18cacd0 2024-09-22T04:03:55.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.563+0000 7fa51cff9640 1 -- 172.21.15.3:0/2230308214 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa510013d70 con 0x18cacd0 2024-09-22T04:03:55.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.563+0000 7fa52b722ec0 1 -- 172.21.15.3:0/2230308214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x18c35f0 con 0x18cacd0 2024-09-22T04:03:55.565 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.564+0000 7fa51cff9640 1 -- 172.21.15.3:0/2230308214 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fa510010400 con 0x18cacd0 2024-09-22T04:03:55.565 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.564+0000 7fa51cff9640 1 --2- 172.21.15.3:0/2230308214 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa50003d360 0x7fa50003f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.565 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.564+0000 7fa51cff9640 1 -- 172.21.15.3:0/2230308214 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(707..707 src has 590..707) ==== 3819+0+0 (crc 0 0 0) 0x7fa510008040 con 0x18cacd0 2024-09-22T04:03:55.565 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.564+0000 7fa52b722ec0 1 --2- 172.21.15.3:0/2230308214 >> v2:172.21.15.123:6806/4206675901 conn(0x194c7f0 0x19f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.565 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.564+0000 7fa52b722ec0 1 -- 172.21.15.3:0/2230308214 --> v2:172.21.15.123:6806/4206675901 -- osd_op(unknown.0.0:1 199.8 199:104778fc:::obj2:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e707) -- 0x194d4e0 con 0x194c7f0 2024-09-22T04:03:55.566 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.564+0000 7fa517fff640 1 --2- 172.21.15.3:0/2230308214 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa50003d360 0x7fa50003f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.566 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.565+0000 7fa517fff640 1 --2- 172.21.15.3:0/2230308214 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa50003d360 0x7fa50003f870 crc :-1 s=READY pgs=3834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.566 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.565+0000 7fa51f7fe640 1 --2- 172.21.15.3:0/2230308214 >> v2:172.21.15.123:6806/4206675901 conn(0x194c7f0 0x19f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:03:55.568 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.568+0000 7fa51f7fe640 1 --2- 172.21.15.3:0/2230308214 >> v2:172.21.15.123:6806/4206675901 conn(0x194c7f0 0x19f3a90 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.595 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.593+0000 7fa51f7fe640 1 -- 172.21.15.3:0/2230308214 <== osd.3 v2:172.21.15.123:6806/4206675901 1 ==== osd_op_reply(1 obj2 [writefull 0~12875] v707'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fa51800a430 con 0x194c7f0 2024-09-22T04:03:55.595 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.593+0000 7fa52b722ec0 1 -- 172.21.15.3:0/2230308214 >> v2:172.21.15.123:6806/4206675901 conn(0x194c7f0 msgr2=0x19f3a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.595 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.593+0000 7fa52b722ec0 1 --2- 172.21.15.3:0/2230308214 >> v2:172.21.15.123:6806/4206675901 conn(0x194c7f0 0x19f3a90 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.595 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.594+0000 7fa52b722ec0 1 -- 172.21.15.3:0/2230308214 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa50003d360 msgr2=0x7fa50003f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.595 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.594+0000 7fa52b722ec0 1 --2- 172.21.15.3:0/2230308214 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa50003d360 0x7fa50003f870 crc :-1 s=READY pgs=3834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.595 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.594+0000 7fa52b722ec0 1 -- 172.21.15.3:0/2230308214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18cacd0 msgr2=0x18c23e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.596 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.594+0000 7fa52b722ec0 1 --2- 172.21.15.3:0/2230308214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18cacd0 0x18c23e0 crc :-1 s=READY pgs=12876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.596 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.594+0000 7fa52b722ec0 1 -- 172.21.15.3:0/2230308214 shutdown_connections 2024-09-22T04:03:55.596 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.594+0000 7fa52b722ec0 1 --2- 172.21.15.3:0/2230308214 >> v2:172.21.15.123:6806/4206675901 conn(0x194c7f0 0x19f3a90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.596 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.594+0000 7fa52b722ec0 1 --2- 172.21.15.3:0/2230308214 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa50003d360 0x7fa50003f870 unknown :-1 s=CLOSED pgs=3834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.596 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.594+0000 7fa52b722ec0 1 --2- 172.21.15.3:0/2230308214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18cacd0 0x18c23e0 unknown :-1 s=CLOSED pgs=12876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.596 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.594+0000 7fa52b722ec0 1 -- 172.21.15.3:0/2230308214 >> 172.21.15.3:0/2230308214 conn(0x17ee770 msgr2=0x18cdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:55.596 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.594+0000 7fa52b722ec0 1 -- 172.21.15.3:0/2230308214 shutdown_connections 2024-09-22T04:03:55.596 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.594+0000 7fa52b722ec0 1 -- 172.21.15.3:0/2230308214 wait complete. 2024-09-22T04:03:55.597 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:03:55.597 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p 7c1c124a-aa8e-411d-9731-1f1389755458 put obj3 /etc/passwd 2024-09-22T04:03:55.634 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.633+0000 7f5ae423fec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.640 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.639+0000 7f5ae423fec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.640 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.639+0000 7f5ae423fec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.641 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.640+0000 7f5ae423fec0 1 Processor -- start 2024-09-22T04:03:55.641 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.640+0000 7f5ae423fec0 1 -- start start 2024-09-22T04:03:55.643 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.641+0000 7f5ae423fec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18e4740 0x18e5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.643 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.641+0000 7f5ae423fec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x17b9520 con 0x18e4740 2024-09-22T04:03:55.643 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.642+0000 7f5ad77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18e4740 0x18e5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.643 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.642+0000 7f5ad77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18e4740 0x18e5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33152/0 (socket says 172.21.15.3:33152) 2024-09-22T04:03:55.644 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.642+0000 7f5ad77fe640 1 -- 172.21.15.3:0/2224375743 learned_addr learned my addr 172.21.15.3:0/2224375743 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:55.644 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.642+0000 7f5ad77fe640 1 -- 172.21.15.3:0/2224375743 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1755e10 con 0x18e4740 2024-09-22T04:03:55.644 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.642+0000 7f5ad77fe640 1 --2- 172.21.15.3:0/2224375743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18e4740 0x18e5dc0 crc :-1 s=READY pgs=12879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41588d8d34095b62 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.644 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.642+0000 7f5ad5ffb640 1 -- 172.21.15.3:0/2224375743 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5ac8032040 con 0x18e4740 2024-09-22T04:03:55.644 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.642+0000 7f5ad5ffb640 1 -- 172.21.15.3:0/2224375743 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5ac8033040 con 0x18e4740 2024-09-22T04:03:55.644 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.642+0000 7f5ad5ffb640 1 -- 172.21.15.3:0/2224375743 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5ac802dcf0 con 0x18e4740 2024-09-22T04:03:55.645 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.643+0000 7f5ae423fec0 1 -- 172.21.15.3:0/2224375743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18e4740 msgr2=0x18e5dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.645 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.643+0000 7f5ae423fec0 1 --2- 172.21.15.3:0/2224375743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18e4740 0x18e5dc0 crc :-1 s=READY pgs=12879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.645 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.643+0000 7f5ae423fec0 1 -- 172.21.15.3:0/2224375743 shutdown_connections 2024-09-22T04:03:55.645 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.643+0000 7f5ae423fec0 1 --2- 172.21.15.3:0/2224375743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18e4740 0x18e5dc0 unknown :-1 s=CLOSED pgs=12879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.645 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.643+0000 7f5ae423fec0 1 -- 172.21.15.3:0/2224375743 >> 172.21.15.3:0/2224375743 conn(0x1779770 msgr2=0x196d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:55.645 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.643+0000 7f5ae423fec0 1 -- 172.21.15.3:0/2224375743 shutdown_connections 2024-09-22T04:03:55.645 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.643+0000 7f5ae423fec0 1 -- 172.21.15.3:0/2224375743 wait complete. 2024-09-22T04:03:55.645 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.644+0000 7f5ae423fec0 1 Processor -- start 2024-09-22T04:03:55.645 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.644+0000 7f5ae423fec0 1 -- start start 2024-09-22T04:03:55.647 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.645+0000 7f5ae423fec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18e4740 0x197c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.647 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.645+0000 7f5ae423fec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x17b7280 con 0x18e4740 2024-09-22T04:03:55.647 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.646+0000 7f5ad77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18e4740 0x197c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.647 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.646+0000 7f5ad77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18e4740 0x197c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33168/0 (socket says 172.21.15.3:33168) 2024-09-22T04:03:55.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.646+0000 7f5ad77fe640 1 -- 172.21.15.3:0/3041434286 learned_addr learned my addr 172.21.15.3:0/3041434286 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:55.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.646+0000 7f5ad77fe640 1 -- 172.21.15.3:0/3041434286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1747920 con 0x18e4740 2024-09-22T04:03:55.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.646+0000 7f5ad77fe640 1 --2- 172.21.15.3:0/3041434286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18e4740 0x197c8c0 crc :-1 s=READY pgs=12880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.646+0000 7f5ad4ff9640 1 -- 172.21.15.3:0/3041434286 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5ac8032040 con 0x18e4740 2024-09-22T04:03:55.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.646+0000 7f5ad4ff9640 1 -- 172.21.15.3:0/3041434286 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5ac802dbc0 con 0x18e4740 2024-09-22T04:03:55.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.646+0000 7f5ae423fec0 1 -- 172.21.15.3:0/3041434286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1747380 con 0x18e4740 2024-09-22T04:03:55.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.646+0000 7f5ad4ff9640 1 -- 172.21.15.3:0/3041434286 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5ac8035b00 con 0x18e4740 2024-09-22T04:03:55.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.646+0000 7f5ae423fec0 1 -- 172.21.15.3:0/3041434286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x18d2d60 con 0x18e4740 2024-09-22T04:03:55.649 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.647+0000 7f5ad4ff9640 1 -- 172.21.15.3:0/3041434286 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f5ac8003040 con 0x18e4740 2024-09-22T04:03:55.649 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.647+0000 7f5ad4ff9640 1 --2- 172.21.15.3:0/3041434286 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5ab803d310 0x7f5ab803f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.649 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.647+0000 7f5ad4ff9640 1 -- 172.21.15.3:0/3041434286 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(707..707 src has 590..707) ==== 3819+0+0 (crc 0 0 0) 0x7f5ac80730b0 con 0x18e4740 2024-09-22T04:03:55.649 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.648+0000 7f5ae423fec0 1 --2- 172.21.15.3:0/3041434286 >> v2:172.21.15.3:6801/2889521405 conn(0x18e7500 0x18e2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.649 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.648+0000 7f5ae423fec0 1 -- 172.21.15.3:0/3041434286 --> v2:172.21.15.3:6801/2889521405 -- osd_op(unknown.0.0:1 199.1 199:8dd16f86:::obj3:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e707) -- 0x18d5b30 con 0x18e7500 2024-09-22T04:03:55.649 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.648+0000 7f5ad6ffd640 1 --2- 172.21.15.3:0/3041434286 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5ab803d310 0x7f5ab803f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.650 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.648+0000 7f5ad6ffd640 1 --2- 172.21.15.3:0/3041434286 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5ab803d310 0x7f5ab803f820 crc :-1 s=READY pgs=3835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.650 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.649+0000 7f5adcb9c640 1 --2- 172.21.15.3:0/3041434286 >> v2:172.21.15.3:6801/2889521405 conn(0x18e7500 0x18e2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:03:55.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.652+0000 7f5adcb9c640 1 --2- 172.21.15.3:0/3041434286 >> v2:172.21.15.3:6801/2889521405 conn(0x18e7500 0x18e2ee0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.686+0000 7f5adcb9c640 1 -- 172.21.15.3:0/3041434286 <== osd.0 v2:172.21.15.3:6801/2889521405 1 ==== osd_op_reply(1 obj3 [writefull 0~12875] v707'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5ad000c430 con 0x18e7500 2024-09-22T04:03:55.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.686+0000 7f5ae423fec0 1 -- 172.21.15.3:0/3041434286 >> v2:172.21.15.3:6801/2889521405 conn(0x18e7500 msgr2=0x18e2ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.686+0000 7f5ae423fec0 1 --2- 172.21.15.3:0/3041434286 >> v2:172.21.15.3:6801/2889521405 conn(0x18e7500 0x18e2ee0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.686+0000 7f5ae423fec0 1 -- 172.21.15.3:0/3041434286 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5ab803d310 msgr2=0x7f5ab803f820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.686+0000 7f5ae423fec0 1 --2- 172.21.15.3:0/3041434286 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5ab803d310 0x7f5ab803f820 crc :-1 s=READY pgs=3835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.686+0000 7f5ae423fec0 1 -- 172.21.15.3:0/3041434286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18e4740 msgr2=0x197c8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.686+0000 7f5ae423fec0 1 --2- 172.21.15.3:0/3041434286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18e4740 0x197c8c0 crc :-1 s=READY pgs=12880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.689 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.687+0000 7f5ae423fec0 1 -- 172.21.15.3:0/3041434286 shutdown_connections 2024-09-22T04:03:55.689 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.687+0000 7f5ae423fec0 1 --2- 172.21.15.3:0/3041434286 >> v2:172.21.15.3:6801/2889521405 conn(0x18e7500 0x18e2ee0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.689 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.687+0000 7f5ae423fec0 1 --2- 172.21.15.3:0/3041434286 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f5ab803d310 0x7f5ab803f820 unknown :-1 s=CLOSED pgs=3835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.689 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.687+0000 7f5ae423fec0 1 --2- 172.21.15.3:0/3041434286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18e4740 0x197c8c0 unknown :-1 s=CLOSED pgs=12880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.689 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.687+0000 7f5ae423fec0 1 -- 172.21.15.3:0/3041434286 >> 172.21.15.3:0/3041434286 conn(0x1779770 msgr2=0x196d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:55.689 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.687+0000 7f5ae423fec0 1 -- 172.21.15.3:0/3041434286 shutdown_connections 2024-09-22T04:03:55.689 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.687+0000 7f5ae423fec0 1 -- 172.21.15.3:0/3041434286 wait complete. 2024-09-22T04:03:55.690 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:03:55.690 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p 7c1c124a-aa8e-411d-9731-1f1389755458 put obj4 /etc/passwd 2024-09-22T04:03:55.727 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.726+0000 7fb0f047dec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.733 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.732+0000 7fb0f047dec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.733 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.732+0000 7fb0f047dec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.734 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.733+0000 7fb0f047dec0 1 Processor -- start 2024-09-22T04:03:55.734 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.733+0000 7fb0f047dec0 1 -- start start 2024-09-22T04:03:55.736 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.734+0000 7fb0f047dec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x9cfd70 0xae9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.736 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.734+0000 7fb0f047dec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x934520 con 0x9cfd70 2024-09-22T04:03:55.736 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.735+0000 7fb0e37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x9cfd70 0xae9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.736 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.735+0000 7fb0e37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x9cfd70 0xae9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33178/0 (socket says 172.21.15.3:33178) 2024-09-22T04:03:55.737 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.735+0000 7fb0e37fe640 1 -- 172.21.15.3:0/1181090782 learned_addr learned my addr 172.21.15.3:0/1181090782 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:55.737 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.735+0000 7fb0e37fe640 1 -- 172.21.15.3:0/1181090782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x8d0e10 con 0x9cfd70 2024-09-22T04:03:55.737 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.735+0000 7fb0e37fe640 1 --2- 172.21.15.3:0/1181090782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x9cfd70 0xae9620 crc :-1 s=READY pgs=12883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f545b7d0933aa17e server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.737 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.735+0000 7fb0e27fc640 1 -- 172.21.15.3:0/1181090782 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb0d402dc90 con 0x9cfd70 2024-09-22T04:03:55.737 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.735+0000 7fb0e27fc640 1 -- 172.21.15.3:0/1181090782 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb0d402de90 con 0x9cfd70 2024-09-22T04:03:55.737 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.735+0000 7fb0e27fc640 1 -- 172.21.15.3:0/1181090782 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb0d400a170 con 0x9cfd70 2024-09-22T04:03:55.738 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.736+0000 7fb0f047dec0 1 -- 172.21.15.3:0/1181090782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x9cfd70 msgr2=0xae9620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.738 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.736+0000 7fb0f047dec0 1 --2- 172.21.15.3:0/1181090782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x9cfd70 0xae9620 crc :-1 s=READY pgs=12883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.738 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.736+0000 7fb0f047dec0 1 -- 172.21.15.3:0/1181090782 shutdown_connections 2024-09-22T04:03:55.739 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.736+0000 7fb0f047dec0 1 --2- 172.21.15.3:0/1181090782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x9cfd70 0xae9620 unknown :-1 s=CLOSED pgs=12883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.739 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.736+0000 7fb0f047dec0 1 -- 172.21.15.3:0/1181090782 >> 172.21.15.3:0/1181090782 conn(0x8f4770 msgr2=0x9d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:55.739 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.736+0000 7fb0f047dec0 1 -- 172.21.15.3:0/1181090782 shutdown_connections 2024-09-22T04:03:55.739 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.736+0000 7fb0f047dec0 1 -- 172.21.15.3:0/1181090782 wait complete. 2024-09-22T04:03:55.739 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.737+0000 7fb0f047dec0 1 Processor -- start 2024-09-22T04:03:55.739 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.737+0000 7fb0f047dec0 1 -- start start 2024-09-22T04:03:55.740 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.738+0000 7fb0f047dec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x9cfd70 0x9cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.740 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.738+0000 7fb0f047dec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x932280 con 0x9cfd70 2024-09-22T04:03:55.740 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.738+0000 7fb0e37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x9cfd70 0x9cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.740 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.738+0000 7fb0e37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x9cfd70 0x9cc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33180/0 (socket says 172.21.15.3:33180) 2024-09-22T04:03:55.741 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.738+0000 7fb0e37fe640 1 -- 172.21.15.3:0/207591116 learned_addr learned my addr 172.21.15.3:0/207591116 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:55.741 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.739+0000 7fb0e37fe640 1 -- 172.21.15.3:0/207591116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x8c2920 con 0x9cfd70 2024-09-22T04:03:55.741 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.739+0000 7fb0e37fe640 1 --2- 172.21.15.3:0/207591116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x9cfd70 0x9cc080 crc :-1 s=READY pgs=12884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.741 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.739+0000 7fb0e17fa640 1 -- 172.21.15.3:0/207591116 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb0d4009620 con 0x9cfd70 2024-09-22T04:03:55.741 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.739+0000 7fb0e17fa640 1 -- 172.21.15.3:0/207591116 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb0d4009780 con 0x9cfd70 2024-09-22T04:03:55.741 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.739+0000 7fb0f047dec0 1 -- 172.21.15.3:0/207591116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x8c2380 con 0x9cfd70 2024-09-22T04:03:55.741 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.739+0000 7fb0e17fa640 1 -- 172.21.15.3:0/207591116 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb0d4009a20 con 0x9cfd70 2024-09-22T04:03:55.741 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.739+0000 7fb0f047dec0 1 -- 172.21.15.3:0/207591116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x9ce280 con 0x9cfd70 2024-09-22T04:03:55.742 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.740+0000 7fb0e17fa640 1 -- 172.21.15.3:0/207591116 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fb0d4009c20 con 0x9cfd70 2024-09-22T04:03:55.742 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.740+0000 7fb0e17fa640 1 --2- 172.21.15.3:0/207591116 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fb0c403d360 0x7fb0c403f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.742 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.740+0000 7fb0e17fa640 1 -- 172.21.15.3:0/207591116 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(707..707 src has 590..707) ==== 3819+0+0 (crc 0 0 0) 0x7fb0d4032040 con 0x9cfd70 2024-09-22T04:03:55.742 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.741+0000 7fb0daffd640 1 --2- 172.21.15.3:0/207591116 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fb0c403d360 0x7fb0c403f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.742 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.741+0000 7fb0f047dec0 1 --2- 172.21.15.3:0/207591116 >> v2:172.21.15.3:6802/2784566484 conn(0xa62a70 0xa5b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.742 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.741+0000 7fb0f047dec0 1 -- 172.21.15.3:0/207591116 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:1 199.0 199:0ee9ae15:::obj4:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e707) -- 0xa4bb50 con 0xa62a70 2024-09-22T04:03:55.742 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.741+0000 7fb0daffd640 1 --2- 172.21.15.3:0/207591116 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fb0c403d360 0x7fb0c403f870 crc :-1 s=READY pgs=3836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.743 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.741+0000 7fb0e8d9c640 1 --2- 172.21.15.3:0/207591116 >> v2:172.21.15.3:6802/2784566484 conn(0xa62a70 0xa5b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:03:55.745 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.744+0000 7fb0e8d9c640 1 --2- 172.21.15.3:0/207591116 >> v2:172.21.15.3:6802/2784566484 conn(0xa62a70 0xa5b5e0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.771 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.769+0000 7fb0e8d9c640 1 -- 172.21.15.3:0/207591116 <== osd.1 v2:172.21.15.3:6802/2784566484 1 ==== osd_op_reply(1 obj4 [writefull 0~12875] v707'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fb0dc00a430 con 0xa62a70 2024-09-22T04:03:55.771 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.769+0000 7fb0f047dec0 1 -- 172.21.15.3:0/207591116 >> v2:172.21.15.3:6802/2784566484 conn(0xa62a70 msgr2=0xa5b5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.771 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.769+0000 7fb0f047dec0 1 --2- 172.21.15.3:0/207591116 >> v2:172.21.15.3:6802/2784566484 conn(0xa62a70 0xa5b5e0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.771 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.769+0000 7fb0f047dec0 1 -- 172.21.15.3:0/207591116 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fb0c403d360 msgr2=0x7fb0c403f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.771 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.769+0000 7fb0f047dec0 1 --2- 172.21.15.3:0/207591116 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fb0c403d360 0x7fb0c403f870 crc :-1 s=READY pgs=3836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.771 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.769+0000 7fb0f047dec0 1 -- 172.21.15.3:0/207591116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x9cfd70 msgr2=0x9cc080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.771 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.769+0000 7fb0f047dec0 1 --2- 172.21.15.3:0/207591116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x9cfd70 0x9cc080 crc :-1 s=READY pgs=12884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.771 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.769+0000 7fb0f047dec0 1 -- 172.21.15.3:0/207591116 shutdown_connections 2024-09-22T04:03:55.771 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.769+0000 7fb0f047dec0 1 --2- 172.21.15.3:0/207591116 >> v2:172.21.15.3:6802/2784566484 conn(0xa62a70 0xa5b5e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.771 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.769+0000 7fb0f047dec0 1 --2- 172.21.15.3:0/207591116 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fb0c403d360 0x7fb0c403f870 unknown :-1 s=CLOSED pgs=3836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.771 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.769+0000 7fb0f047dec0 1 --2- 172.21.15.3:0/207591116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x9cfd70 0x9cc080 unknown :-1 s=CLOSED pgs=12884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.772 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.769+0000 7fb0f047dec0 1 -- 172.21.15.3:0/207591116 >> 172.21.15.3:0/207591116 conn(0x8f4770 msgr2=0x9d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:55.772 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.769+0000 7fb0f047dec0 1 -- 172.21.15.3:0/207591116 shutdown_connections 2024-09-22T04:03:55.772 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.769+0000 7fb0f047dec0 1 -- 172.21.15.3:0/207591116 wait complete. 2024-09-22T04:03:55.773 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:03:55.773 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p 7c1c124a-aa8e-411d-9731-1f1389755458 put obj5 /etc/passwd 2024-09-22T04:03:55.811 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.810+0000 7f26a1a84ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.816 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.815+0000 7f26a1a84ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.817 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.816+0000 7f26a1a84ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.818 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.816+0000 7f26a1a84ec0 1 Processor -- start 2024-09-22T04:03:55.818 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.817+0000 7f26a1a84ec0 1 -- start start 2024-09-22T04:03:55.820 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.818+0000 7f26a1a84ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x15f9f80 0x171b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.820 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.818+0000 7f26a1a84ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x1565520 con 0x15f9f80 2024-09-22T04:03:55.820 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.818+0000 7f2699b9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x15f9f80 0x171b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.820 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.818+0000 7f2699b9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x15f9f80 0x171b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33208/0 (socket says 172.21.15.3:33208) 2024-09-22T04:03:55.820 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.818+0000 7f2699b9b640 1 -- 172.21.15.3:0/3219869819 learned_addr learned my addr 172.21.15.3:0/3219869819 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:55.820 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.818+0000 7f2699b9b640 1 -- 172.21.15.3:0/3219869819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1501e10 con 0x15f9f80 2024-09-22T04:03:55.820 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.819+0000 7f2699b9b640 1 --2- 172.21.15.3:0/3219869819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x15f9f80 0x171b8a0 crc :-1 s=READY pgs=12887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03849b91e01ac03 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.820 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.819+0000 7f26937fe640 1 -- 172.21.15.3:0/3219869819 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f268402db90 con 0x15f9f80 2024-09-22T04:03:55.821 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.819+0000 7f26937fe640 1 -- 172.21.15.3:0/3219869819 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2684037370 con 0x15f9f80 2024-09-22T04:03:55.821 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.819+0000 7f26937fe640 1 -- 172.21.15.3:0/3219869819 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f26840374e0 con 0x15f9f80 2024-09-22T04:03:55.822 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.819+0000 7f26a1a84ec0 1 -- 172.21.15.3:0/3219869819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x15f9f80 msgr2=0x171b8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.822 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.819+0000 7f26a1a84ec0 1 --2- 172.21.15.3:0/3219869819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x15f9f80 0x171b8a0 crc :-1 s=READY pgs=12887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.822 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.819+0000 7f26a1a84ec0 1 -- 172.21.15.3:0/3219869819 shutdown_connections 2024-09-22T04:03:55.822 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.819+0000 7f26a1a84ec0 1 --2- 172.21.15.3:0/3219869819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x15f9f80 0x171b8a0 unknown :-1 s=CLOSED pgs=12887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.822 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.819+0000 7f26a1a84ec0 1 -- 172.21.15.3:0/3219869819 >> 172.21.15.3:0/3219869819 conn(0x1525770 msgr2=0x15fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:55.822 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.819+0000 7f26a1a84ec0 1 -- 172.21.15.3:0/3219869819 shutdown_connections 2024-09-22T04:03:55.822 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.819+0000 7f26a1a84ec0 1 -- 172.21.15.3:0/3219869819 wait complete. 2024-09-22T04:03:55.822 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.820+0000 7f26a1a84ec0 1 Processor -- start 2024-09-22T04:03:55.822 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.820+0000 7f26a1a84ec0 1 -- start start 2024-09-22T04:03:55.823 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.822+0000 7f26a1a84ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x15f9f80 0x1693410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.823 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.822+0000 7f26a1a84ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x1563280 con 0x15f9f80 2024-09-22T04:03:55.823 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.822+0000 7f2699b9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x15f9f80 0x1693410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.823 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.822+0000 7f2699b9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x15f9f80 0x1693410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33212/0 (socket says 172.21.15.3:33212) 2024-09-22T04:03:55.823 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.822+0000 7f2699b9b640 1 -- 172.21.15.3:0/2076006250 learned_addr learned my addr 172.21.15.3:0/2076006250 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:55.824 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.822+0000 7f2699b9b640 1 -- 172.21.15.3:0/2076006250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x14f3920 con 0x15f9f80 2024-09-22T04:03:55.824 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.823+0000 7f2699b9b640 1 --2- 172.21.15.3:0/2076006250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x15f9f80 0x1693410 crc :-1 s=READY pgs=12888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.824 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.823+0000 7f26927fc640 1 -- 172.21.15.3:0/2076006250 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f26840057c0 con 0x15f9f80 2024-09-22T04:03:55.824 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.823+0000 7f26927fc640 1 -- 172.21.15.3:0/2076006250 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f26840374e0 con 0x15f9f80 2024-09-22T04:03:55.825 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.823+0000 7f26a1a84ec0 1 -- 172.21.15.3:0/2076006250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x14f3380 con 0x15f9f80 2024-09-22T04:03:55.825 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.823+0000 7f26927fc640 1 -- 172.21.15.3:0/2076006250 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f268402dcc0 con 0x15f9f80 2024-09-22T04:03:55.825 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.823+0000 7f26a1a84ec0 1 -- 172.21.15.3:0/2076006250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1681200 con 0x15f9f80 2024-09-22T04:03:55.826 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.824+0000 7f26927fc640 1 -- 172.21.15.3:0/2076006250 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f2684039450 con 0x15f9f80 2024-09-22T04:03:55.826 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.824+0000 7f26927fc640 1 --2- 172.21.15.3:0/2076006250 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f268003d360 0x7f268003f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.826 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.824+0000 7f26927fc640 1 -- 172.21.15.3:0/2076006250 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(707..707 src has 590..707) ==== 3819+0+0 (crc 0 0 0) 0x7f2684032040 con 0x15f9f80 2024-09-22T04:03:55.826 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.824+0000 7f26a1a84ec0 1 --2- 172.21.15.3:0/2076006250 >> v2:172.21.15.123:6806/4206675901 conn(0x16020f0 0x167ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.826 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.824+0000 7f26a1a84ec0 1 -- 172.21.15.3:0/2076006250 --> v2:172.21.15.123:6806/4206675901 -- osd_op(unknown.0.0:1 199.3 199:c52c9666:::obj5:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e707) -- 0x1683180 con 0x16020f0 2024-09-22T04:03:55.826 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.824+0000 7f269939a640 1 --2- 172.21.15.3:0/2076006250 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f268003d360 0x7f268003f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.826 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.825+0000 7f269939a640 1 --2- 172.21.15.3:0/2076006250 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f268003d360 0x7f268003f870 crc :-1 s=READY pgs=3837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.826 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.825+0000 7f269a39c640 1 --2- 172.21.15.3:0/2076006250 >> v2:172.21.15.123:6806/4206675901 conn(0x16020f0 0x167ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:03:55.830 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.829+0000 7f269a39c640 1 --2- 172.21.15.3:0/2076006250 >> v2:172.21.15.123:6806/4206675901 conn(0x16020f0 0x167ff50 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.861 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.859+0000 7f269a39c640 1 -- 172.21.15.3:0/2076006250 <== osd.3 v2:172.21.15.123:6806/4206675901 1 ==== osd_op_reply(1 obj5 [writefull 0~12875] v707'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f268c00a430 con 0x16020f0 2024-09-22T04:03:55.861 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.859+0000 7f26a1a84ec0 1 -- 172.21.15.3:0/2076006250 >> v2:172.21.15.123:6806/4206675901 conn(0x16020f0 msgr2=0x167ff50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.861 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.859+0000 7f26a1a84ec0 1 --2- 172.21.15.3:0/2076006250 >> v2:172.21.15.123:6806/4206675901 conn(0x16020f0 0x167ff50 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.861 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.859+0000 7f26a1a84ec0 1 -- 172.21.15.3:0/2076006250 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f268003d360 msgr2=0x7f268003f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.861 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.859+0000 7f26a1a84ec0 1 --2- 172.21.15.3:0/2076006250 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f268003d360 0x7f268003f870 crc :-1 s=READY pgs=3837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.861 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.859+0000 7f26a1a84ec0 1 -- 172.21.15.3:0/2076006250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x15f9f80 msgr2=0x1693410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.861 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.859+0000 7f26a1a84ec0 1 --2- 172.21.15.3:0/2076006250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x15f9f80 0x1693410 crc :-1 s=READY pgs=12888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.862 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.859+0000 7f26a1a84ec0 1 -- 172.21.15.3:0/2076006250 shutdown_connections 2024-09-22T04:03:55.862 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.859+0000 7f26a1a84ec0 1 --2- 172.21.15.3:0/2076006250 >> v2:172.21.15.123:6806/4206675901 conn(0x16020f0 0x167ff50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.862 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.859+0000 7f26a1a84ec0 1 --2- 172.21.15.3:0/2076006250 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f268003d360 0x7f268003f870 unknown :-1 s=CLOSED pgs=3837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.862 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.859+0000 7f26a1a84ec0 1 --2- 172.21.15.3:0/2076006250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x15f9f80 0x1693410 unknown :-1 s=CLOSED pgs=12888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.862 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.859+0000 7f26a1a84ec0 1 -- 172.21.15.3:0/2076006250 >> 172.21.15.3:0/2076006250 conn(0x1525770 msgr2=0x15fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:55.862 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.859+0000 7f26a1a84ec0 1 -- 172.21.15.3:0/2076006250 shutdown_connections 2024-09-22T04:03:55.862 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.859+0000 7f26a1a84ec0 1 -- 172.21.15.3:0/2076006250 wait complete. 2024-09-22T04:03:55.863 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:03:55.863 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p 7c1c124a-aa8e-411d-9731-1f1389755458 put obj6 /etc/passwd 2024-09-22T04:03:55.901 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.900+0000 7fa55fae9ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.906 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.905+0000 7fa55fae9ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.906 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.905+0000 7fa55fae9ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.907 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.906+0000 7fa55fae9ec0 1 Processor -- start 2024-09-22T04:03:55.907 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.906+0000 7fa55fae9ec0 1 -- start start 2024-09-22T04:03:55.910 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.908+0000 7fa55fae9ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2436f80 0x24b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.910 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.908+0000 7fa55fae9ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x239b520 con 0x2436f80 2024-09-22T04:03:55.910 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.908+0000 7fa552ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2436f80 0x24b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.910 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.908+0000 7fa552ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2436f80 0x24b56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33226/0 (socket says 172.21.15.3:33226) 2024-09-22T04:03:55.910 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.908+0000 7fa552ffd640 1 -- 172.21.15.3:0/3109550379 learned_addr learned my addr 172.21.15.3:0/3109550379 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:55.910 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.908+0000 7fa552ffd640 1 -- 172.21.15.3:0/3109550379 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x2337e10 con 0x2436f80 2024-09-22T04:03:55.910 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.909+0000 7fa552ffd640 1 --2- 172.21.15.3:0/3109550379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2436f80 0x24b56b0 crc :-1 s=READY pgs=12891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781ea7dee8c566eb server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.911 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.909+0000 7fa551ffb640 1 -- 172.21.15.3:0/3109550379 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa54402db90 con 0x2436f80 2024-09-22T04:03:55.911 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.909+0000 7fa551ffb640 1 -- 172.21.15.3:0/3109550379 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa544037370 con 0x2436f80 2024-09-22T04:03:55.911 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.909+0000 7fa551ffb640 1 -- 172.21.15.3:0/3109550379 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa5440374e0 con 0x2436f80 2024-09-22T04:03:55.912 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.909+0000 7fa55fae9ec0 1 -- 172.21.15.3:0/3109550379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2436f80 msgr2=0x24b56b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.912 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.909+0000 7fa55fae9ec0 1 --2- 172.21.15.3:0/3109550379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2436f80 0x24b56b0 crc :-1 s=READY pgs=12891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.912 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.909+0000 7fa55fae9ec0 1 -- 172.21.15.3:0/3109550379 shutdown_connections 2024-09-22T04:03:55.912 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.909+0000 7fa55fae9ec0 1 --2- 172.21.15.3:0/3109550379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2436f80 0x24b56b0 unknown :-1 s=CLOSED pgs=12891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.912 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.909+0000 7fa55fae9ec0 1 -- 172.21.15.3:0/3109550379 >> 172.21.15.3:0/3109550379 conn(0x235b770 msgr2=0x2431790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:55.912 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.910+0000 7fa55fae9ec0 1 -- 172.21.15.3:0/3109550379 shutdown_connections 2024-09-22T04:03:55.912 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.910+0000 7fa55fae9ec0 1 -- 172.21.15.3:0/3109550379 wait complete. 2024-09-22T04:03:55.912 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.910+0000 7fa55fae9ec0 1 Processor -- start 2024-09-22T04:03:55.912 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.911+0000 7fa55fae9ec0 1 -- start start 2024-09-22T04:03:55.914 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.912+0000 7fa55fae9ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2436f80 0x24b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.914 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.912+0000 7fa55fae9ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x2399280 con 0x2436f80 2024-09-22T04:03:55.914 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.912+0000 7fa552ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2436f80 0x24b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.914 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.912+0000 7fa552ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2436f80 0x24b8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33232/0 (socket says 172.21.15.3:33232) 2024-09-22T04:03:55.914 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.912+0000 7fa552ffd640 1 -- 172.21.15.3:0/1895637138 learned_addr learned my addr 172.21.15.3:0/1895637138 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:55.914 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.912+0000 7fa552ffd640 1 -- 172.21.15.3:0/1895637138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x2329920 con 0x2436f80 2024-09-22T04:03:55.914 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.912+0000 7fa552ffd640 1 --2- 172.21.15.3:0/1895637138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2436f80 0x24b8f70 crc :-1 s=READY pgs=12892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.914 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.913+0000 7fa550ff9640 1 -- 172.21.15.3:0/1895637138 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa54403e040 con 0x2436f80 2024-09-22T04:03:55.914 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.913+0000 7fa550ff9640 1 -- 172.21.15.3:0/1895637138 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa544037eb0 con 0x2436f80 2024-09-22T04:03:55.914 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.913+0000 7fa55fae9ec0 1 -- 172.21.15.3:0/1895637138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x2329380 con 0x2436f80 2024-09-22T04:03:55.915 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.913+0000 7fa550ff9640 1 -- 172.21.15.3:0/1895637138 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa54402db90 con 0x2436f80 2024-09-22T04:03:55.915 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.913+0000 7fa55fae9ec0 1 -- 172.21.15.3:0/1895637138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x24bcb80 con 0x2436f80 2024-09-22T04:03:55.916 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.914+0000 7fa550ff9640 1 -- 172.21.15.3:0/1895637138 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fa544041040 con 0x2436f80 2024-09-22T04:03:55.916 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.914+0000 7fa550ff9640 1 --2- 172.21.15.3:0/1895637138 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa53403d360 0x7fa53403f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.916 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.914+0000 7fa550ff9640 1 -- 172.21.15.3:0/1895637138 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(707..707 src has 590..707) ==== 3819+0+0 (crc 0 0 0) 0x7fa544032040 con 0x2436f80 2024-09-22T04:03:55.916 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.914+0000 7fa55fae9ec0 1 --2- 172.21.15.3:0/1895637138 >> v2:172.21.15.3:6801/2889521405 conn(0x2437cd0 0x2962a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:55.916 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.914+0000 7fa55fae9ec0 1 -- 172.21.15.3:0/1895637138 --> v2:172.21.15.3:6801/2889521405 -- osd_op(unknown.0.0:1 199.4 199:233a42c1:::obj6:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e707) -- 0x24bc5c0 con 0x2437cd0 2024-09-22T04:03:55.916 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.914+0000 7fa54bfff640 1 --2- 172.21.15.3:0/1895637138 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa53403d360 0x7fa53403f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:55.916 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.915+0000 7fa54bfff640 1 --2- 172.21.15.3:0/1895637138 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa53403d360 0x7fa53403f870 crc :-1 s=READY pgs=3838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.917 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.915+0000 7fa5537fe640 1 --2- 172.21.15.3:0/1895637138 >> v2:172.21.15.3:6801/2889521405 conn(0x2437cd0 0x2962a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:03:55.919 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.919+0000 7fa5537fe640 1 --2- 172.21.15.3:0/1895637138 >> v2:172.21.15.3:6801/2889521405 conn(0x2437cd0 0x2962a60 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:55.951 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.949+0000 7fa5537fe640 1 -- 172.21.15.3:0/1895637138 <== osd.0 v2:172.21.15.3:6801/2889521405 1 ==== osd_op_reply(1 obj6 [writefull 0~12875] v707'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fa54c00a430 con 0x2437cd0 2024-09-22T04:03:55.951 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.949+0000 7fa55fae9ec0 1 -- 172.21.15.3:0/1895637138 >> v2:172.21.15.3:6801/2889521405 conn(0x2437cd0 msgr2=0x2962a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.951 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.949+0000 7fa55fae9ec0 1 --2- 172.21.15.3:0/1895637138 >> v2:172.21.15.3:6801/2889521405 conn(0x2437cd0 0x2962a60 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.952 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.950+0000 7fa55fae9ec0 1 -- 172.21.15.3:0/1895637138 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa53403d360 msgr2=0x7fa53403f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.952 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.950+0000 7fa55fae9ec0 1 --2- 172.21.15.3:0/1895637138 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa53403d360 0x7fa53403f870 crc :-1 s=READY pgs=3838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.952 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.950+0000 7fa55fae9ec0 1 -- 172.21.15.3:0/1895637138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2436f80 msgr2=0x24b8f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:55.952 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.950+0000 7fa55fae9ec0 1 --2- 172.21.15.3:0/1895637138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2436f80 0x24b8f70 crc :-1 s=READY pgs=12892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.952 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.950+0000 7fa55fae9ec0 1 -- 172.21.15.3:0/1895637138 shutdown_connections 2024-09-22T04:03:55.952 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.950+0000 7fa55fae9ec0 1 --2- 172.21.15.3:0/1895637138 >> v2:172.21.15.3:6801/2889521405 conn(0x2437cd0 0x2962a60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.952 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.950+0000 7fa55fae9ec0 1 --2- 172.21.15.3:0/1895637138 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa53403d360 0x7fa53403f870 unknown :-1 s=CLOSED pgs=3838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.952 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.950+0000 7fa55fae9ec0 1 --2- 172.21.15.3:0/1895637138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2436f80 0x24b8f70 unknown :-1 s=CLOSED pgs=12892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:55.952 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.950+0000 7fa55fae9ec0 1 -- 172.21.15.3:0/1895637138 >> 172.21.15.3:0/1895637138 conn(0x235b770 msgr2=0x2431790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:55.952 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.950+0000 7fa55fae9ec0 1 -- 172.21.15.3:0/1895637138 shutdown_connections 2024-09-22T04:03:55.952 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.950+0000 7fa55fae9ec0 1 -- 172.21.15.3:0/1895637138 wait complete. 2024-09-22T04:03:55.954 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:03:55.954 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p 7c1c124a-aa8e-411d-9731-1f1389755458 put obj7 /etc/passwd 2024-09-22T04:03:55.991 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.990+0000 7f816ac38ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.997 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.996+0000 7f816ac38ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.997 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.996+0000 7f816ac38ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:55.998 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.997+0000 7f816ac38ec0 1 Processor -- start 2024-09-22T04:03:55.998 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.997+0000 7f816ac38ec0 1 -- start start 2024-09-22T04:03:56.000 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.998+0000 7f816ac38ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2156190 0x21e7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:56.000 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.998+0000 7f816ac38ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x20bb520 con 0x2156190 2024-09-22T04:03:56.000 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.998+0000 7f8162d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2156190 0x21e7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:56.000 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.998+0000 7f8162d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2156190 0x21e7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33262/0 (socket says 172.21.15.3:33262) 2024-09-22T04:03:56.000 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.998+0000 7f8162d9b640 1 -- 172.21.15.3:0/1891244603 learned_addr learned my addr 172.21.15.3:0/1891244603 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:56.000 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.999+0000 7f8162d9b640 1 -- 172.21.15.3:0/1891244603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x2057e10 con 0x2156190 2024-09-22T04:03:56.000 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.999+0000 7f8162d9b640 1 --2- 172.21.15.3:0/1891244603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2156190 0x21e7eb0 crc :-1 s=READY pgs=12895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4918e4b2de79154 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:56.001 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.999+0000 7f8161598640 1 -- 172.21.15.3:0/1891244603 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f814c035040 con 0x2156190 2024-09-22T04:03:56.001 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.999+0000 7f8161598640 1 -- 172.21.15.3:0/1891244603 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f814c02f9d0 con 0x2156190 2024-09-22T04:03:56.001 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:55.999+0000 7f8161598640 1 -- 172.21.15.3:0/1891244603 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f814c039490 con 0x2156190 2024-09-22T04:03:56.002 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.000+0000 7f816ac38ec0 1 -- 172.21.15.3:0/1891244603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2156190 msgr2=0x21e7eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:56.002 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.000+0000 7f816ac38ec0 1 --2- 172.21.15.3:0/1891244603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2156190 0x21e7eb0 crc :-1 s=READY pgs=12895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.002 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.000+0000 7f816ac38ec0 1 -- 172.21.15.3:0/1891244603 shutdown_connections 2024-09-22T04:03:56.002 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.000+0000 7f816ac38ec0 1 --2- 172.21.15.3:0/1891244603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2156190 0x21e7eb0 unknown :-1 s=CLOSED pgs=12895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.002 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.000+0000 7f816ac38ec0 1 -- 172.21.15.3:0/1891244603 >> 172.21.15.3:0/1891244603 conn(0x207b770 msgr2=0x21544b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:56.002 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.000+0000 7f816ac38ec0 1 -- 172.21.15.3:0/1891244603 shutdown_connections 2024-09-22T04:03:56.002 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.000+0000 7f816ac38ec0 1 -- 172.21.15.3:0/1891244603 wait complete. 2024-09-22T04:03:56.002 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.001+0000 7f816ac38ec0 1 Processor -- start 2024-09-22T04:03:56.002 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.001+0000 7f816ac38ec0 1 -- start start 2024-09-22T04:03:56.004 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.002+0000 7f816ac38ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2156190 0x227c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:56.004 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.002+0000 7f816ac38ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x20b9280 con 0x2156190 2024-09-22T04:03:56.004 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.002+0000 7f8162d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2156190 0x227c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:56.004 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.002+0000 7f8162d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2156190 0x227c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33270/0 (socket says 172.21.15.3:33270) 2024-09-22T04:03:56.004 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.002+0000 7f8162d9b640 1 -- 172.21.15.3:0/2336939092 learned_addr learned my addr 172.21.15.3:0/2336939092 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:56.005 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.003+0000 7f8162d9b640 1 -- 172.21.15.3:0/2336939092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x2049920 con 0x2156190 2024-09-22T04:03:56.005 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.003+0000 7f8162d9b640 1 --2- 172.21.15.3:0/2336939092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2156190 0x227c840 crc :-1 s=READY pgs=12896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:56.005 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.003+0000 7f815b7fe640 1 -- 172.21.15.3:0/2336939092 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f814c035040 con 0x2156190 2024-09-22T04:03:56.005 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.003+0000 7f815b7fe640 1 -- 172.21.15.3:0/2336939092 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f814c03bac0 con 0x2156190 2024-09-22T04:03:56.005 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.003+0000 7f816ac38ec0 1 -- 172.21.15.3:0/2336939092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x2049380 con 0x2156190 2024-09-22T04:03:56.005 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.003+0000 7f815b7fe640 1 -- 172.21.15.3:0/2336939092 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f814c03bd60 con 0x2156190 2024-09-22T04:03:56.005 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.003+0000 7f816ac38ec0 1 -- 172.21.15.3:0/2336939092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x2270320 con 0x2156190 2024-09-22T04:03:56.006 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.004+0000 7f815b7fe640 1 -- 172.21.15.3:0/2336939092 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f814c038040 con 0x2156190 2024-09-22T04:03:56.006 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.004+0000 7f815b7fe640 1 --2- 172.21.15.3:0/2336939092 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f814803d360 0x7f814803f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:56.006 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.004+0000 7f815b7fe640 1 -- 172.21.15.3:0/2336939092 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(707..707 src has 590..707) ==== 3819+0+0 (crc 0 0 0) 0x7f814c075ce0 con 0x2156190 2024-09-22T04:03:56.007 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.005+0000 7f816ac38ec0 1 --2- 172.21.15.3:0/2336939092 >> v2:172.21.15.3:6802/2784566484 conn(0x2156ae0 0x2273f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:56.007 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.005+0000 7f816ac38ec0 1 -- 172.21.15.3:0/2336939092 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:1 199.9 199:9a09113a:::obj7:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e707) -- 0x22737a0 con 0x2156ae0 2024-09-22T04:03:56.007 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.005+0000 7f816259a640 1 --2- 172.21.15.3:0/2336939092 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f814803d360 0x7f814803f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:56.007 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.005+0000 7f816259a640 1 --2- 172.21.15.3:0/2336939092 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f814803d360 0x7f814803f870 crc :-1 s=READY pgs=3839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:56.007 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.005+0000 7f816359c640 1 --2- 172.21.15.3:0/2336939092 >> v2:172.21.15.3:6802/2784566484 conn(0x2156ae0 0x2273f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:03:56.010 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.009+0000 7f816359c640 1 --2- 172.21.15.3:0/2336939092 >> v2:172.21.15.3:6802/2784566484 conn(0x2156ae0 0x2273f20 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:56.036 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.034+0000 7f816359c640 1 -- 172.21.15.3:0/2336939092 <== osd.1 v2:172.21.15.3:6802/2784566484 1 ==== osd_op_reply(1 obj7 [writefull 0~12875] v707'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f815400a430 con 0x2156ae0 2024-09-22T04:03:56.036 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.034+0000 7f816ac38ec0 1 -- 172.21.15.3:0/2336939092 >> v2:172.21.15.3:6802/2784566484 conn(0x2156ae0 msgr2=0x2273f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:56.036 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.034+0000 7f816ac38ec0 1 --2- 172.21.15.3:0/2336939092 >> v2:172.21.15.3:6802/2784566484 conn(0x2156ae0 0x2273f20 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.036 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.034+0000 7f816ac38ec0 1 -- 172.21.15.3:0/2336939092 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f814803d360 msgr2=0x7f814803f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:56.036 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.034+0000 7f816ac38ec0 1 --2- 172.21.15.3:0/2336939092 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f814803d360 0x7f814803f870 crc :-1 s=READY pgs=3839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.036 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.034+0000 7f816ac38ec0 1 -- 172.21.15.3:0/2336939092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2156190 msgr2=0x227c840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:56.036 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.034+0000 7f816ac38ec0 1 --2- 172.21.15.3:0/2336939092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2156190 0x227c840 crc :-1 s=READY pgs=12896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.036 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.034+0000 7f816ac38ec0 1 -- 172.21.15.3:0/2336939092 shutdown_connections 2024-09-22T04:03:56.036 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.034+0000 7f816ac38ec0 1 --2- 172.21.15.3:0/2336939092 >> v2:172.21.15.3:6802/2784566484 conn(0x2156ae0 0x2273f20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.036 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.034+0000 7f816ac38ec0 1 --2- 172.21.15.3:0/2336939092 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f814803d360 0x7f814803f870 unknown :-1 s=CLOSED pgs=3839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.036 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.034+0000 7f816ac38ec0 1 --2- 172.21.15.3:0/2336939092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x2156190 0x227c840 unknown :-1 s=CLOSED pgs=12896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.037 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.034+0000 7f816ac38ec0 1 -- 172.21.15.3:0/2336939092 >> 172.21.15.3:0/2336939092 conn(0x207b770 msgr2=0x2154060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:56.037 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.034+0000 7f816ac38ec0 1 -- 172.21.15.3:0/2336939092 shutdown_connections 2024-09-22T04:03:56.037 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.034+0000 7f816ac38ec0 1 -- 172.21.15.3:0/2336939092 wait complete. 2024-09-22T04:03:56.038 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:03:56.038 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p 7c1c124a-aa8e-411d-9731-1f1389755458 put obj8 /etc/passwd 2024-09-22T04:03:56.076 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.074+0000 7fc7348efec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:56.082 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.080+0000 7fc7348efec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:56.082 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.080+0000 7fc7348efec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:56.082 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.081+0000 7fc7348efec0 1 Processor -- start 2024-09-22T04:03:56.082 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.081+0000 7fc7348efec0 1 -- start start 2024-09-22T04:03:56.084 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.083+0000 7fc7348efec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18818b0 0x1885730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:56.084 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.083+0000 7fc7348efec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x17ee520 con 0x18818b0 2024-09-22T04:03:56.085 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.083+0000 7fc72cc3f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18818b0 0x1885730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:56.085 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.083+0000 7fc72cc3f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18818b0 0x1885730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33294/0 (socket says 172.21.15.3:33294) 2024-09-22T04:03:56.085 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.083+0000 7fc72cc3f640 1 -- 172.21.15.3:0/4037971527 learned_addr learned my addr 172.21.15.3:0/4037971527 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:56.085 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.083+0000 7fc72cc3f640 1 -- 172.21.15.3:0/4037971527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x178ae10 con 0x18818b0 2024-09-22T04:03:56.085 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.083+0000 7fc72cc3f640 1 --2- 172.21.15.3:0/4037971527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18818b0 0x1885730 crc :-1 s=READY pgs=12899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81260ce9cd9d8803 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:56.085 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.083+0000 7fc7267fc640 1 -- 172.21.15.3:0/4037971527 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc71800c490 con 0x18818b0 2024-09-22T04:03:56.085 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.083+0000 7fc7267fc640 1 -- 172.21.15.3:0/4037971527 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc71800c5f0 con 0x18818b0 2024-09-22T04:03:56.086 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.083+0000 7fc7267fc640 1 -- 172.21.15.3:0/4037971527 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc71800c890 con 0x18818b0 2024-09-22T04:03:56.087 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.084+0000 7fc7348efec0 1 -- 172.21.15.3:0/4037971527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18818b0 msgr2=0x1885730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:56.087 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.084+0000 7fc7348efec0 1 --2- 172.21.15.3:0/4037971527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18818b0 0x1885730 crc :-1 s=READY pgs=12899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.087 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.084+0000 7fc7348efec0 1 -- 172.21.15.3:0/4037971527 shutdown_connections 2024-09-22T04:03:56.087 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.084+0000 7fc7348efec0 1 --2- 172.21.15.3:0/4037971527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18818b0 0x1885730 unknown :-1 s=CLOSED pgs=12899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.088 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.084+0000 7fc7348efec0 1 -- 172.21.15.3:0/4037971527 >> 172.21.15.3:0/4037971527 conn(0x17ae770 msgr2=0x188c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:56.088 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.084+0000 7fc7348efec0 1 -- 172.21.15.3:0/4037971527 shutdown_connections 2024-09-22T04:03:56.088 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.084+0000 7fc7348efec0 1 -- 172.21.15.3:0/4037971527 wait complete. 2024-09-22T04:03:56.088 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.085+0000 7fc7348efec0 1 Processor -- start 2024-09-22T04:03:56.088 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.085+0000 7fc7348efec0 1 -- start start 2024-09-22T04:03:56.089 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.087+0000 7fc7348efec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18818b0 0x1906240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:56.089 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.087+0000 7fc7348efec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x17ec280 con 0x18818b0 2024-09-22T04:03:56.089 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.087+0000 7fc72cc3f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18818b0 0x1906240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:56.089 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.087+0000 7fc72cc3f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18818b0 0x1906240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33310/0 (socket says 172.21.15.3:33310) 2024-09-22T04:03:56.089 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.087+0000 7fc72cc3f640 1 -- 172.21.15.3:0/872714475 learned_addr learned my addr 172.21.15.3:0/872714475 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:56.090 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.087+0000 7fc72cc3f640 1 -- 172.21.15.3:0/872714475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x177c920 con 0x18818b0 2024-09-22T04:03:56.090 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.087+0000 7fc72cc3f640 1 --2- 172.21.15.3:0/872714475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18818b0 0x1906240 crc :-1 s=READY pgs=12900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:56.090 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.087+0000 7fc7257fa640 1 -- 172.21.15.3:0/872714475 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc718016040 con 0x18818b0 2024-09-22T04:03:56.090 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.087+0000 7fc7257fa640 1 -- 172.21.15.3:0/872714475 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc718011b00 con 0x18818b0 2024-09-22T04:03:56.090 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.087+0000 7fc7348efec0 1 -- 172.21.15.3:0/872714475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x177c380 con 0x18818b0 2024-09-22T04:03:56.090 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.087+0000 7fc7257fa640 1 -- 172.21.15.3:0/872714475 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc718011da0 con 0x18818b0 2024-09-22T04:03:56.090 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.087+0000 7fc7348efec0 1 -- 172.21.15.3:0/872714475 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x190d860 con 0x18818b0 2024-09-22T04:03:56.090 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.088+0000 7fc7257fa640 1 -- 172.21.15.3:0/872714475 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fc71800e440 con 0x18818b0 2024-09-22T04:03:56.091 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.089+0000 7fc7257fa640 1 --2- 172.21.15.3:0/872714475 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc71403d360 0x7fc71403f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:56.091 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.089+0000 7fc7257fa640 1 -- 172.21.15.3:0/872714475 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(707..707 src has 590..707) ==== 3819+0+0 (crc 0 0 0) 0x7fc718050740 con 0x18818b0 2024-09-22T04:03:56.091 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.089+0000 7fc7348efec0 1 --2- 172.21.15.3:0/872714475 >> v2:172.21.15.123:6813/891966684 conn(0x191e770 0x19b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:56.091 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.089+0000 7fc7348efec0 1 -- 172.21.15.3:0/872714475 --> v2:172.21.15.123:6813/891966684 -- osd_op(unknown.0.0:1 199.5 199:a557efb1:::obj8:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e707) -- 0x190cfd0 con 0x191e770 2024-09-22T04:03:56.091 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.089+0000 7fc7277fe640 1 --2- 172.21.15.3:0/872714475 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc71403d360 0x7fc71403f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:56.091 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.089+0000 7fc7277fe640 1 --2- 172.21.15.3:0/872714475 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc71403d360 0x7fc71403f870 crc :-1 s=READY pgs=3840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:56.091 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.089+0000 7fc72d440640 1 --2- 172.21.15.3:0/872714475 >> v2:172.21.15.123:6813/891966684 conn(0x191e770 0x19b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:03:56.093 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.092+0000 7fc72d440640 1 --2- 172.21.15.3:0/872714475 >> v2:172.21.15.123:6813/891966684 conn(0x191e770 0x19b3990 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:56.118 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.116+0000 7fc72d440640 1 -- 172.21.15.3:0/872714475 <== osd.2 v2:172.21.15.123:6813/891966684 1 ==== osd_op_reply(1 obj8 [writefull 0~12875] v707'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fc72000a430 con 0x191e770 2024-09-22T04:03:56.118 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.116+0000 7fc7348efec0 1 -- 172.21.15.3:0/872714475 >> v2:172.21.15.123:6813/891966684 conn(0x191e770 msgr2=0x19b3990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:56.118 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.116+0000 7fc7348efec0 1 --2- 172.21.15.3:0/872714475 >> v2:172.21.15.123:6813/891966684 conn(0x191e770 0x19b3990 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.118 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.116+0000 7fc7348efec0 1 -- 172.21.15.3:0/872714475 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc71403d360 msgr2=0x7fc71403f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:56.118 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.116+0000 7fc7348efec0 1 --2- 172.21.15.3:0/872714475 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc71403d360 0x7fc71403f870 crc :-1 s=READY pgs=3840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.118 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.116+0000 7fc7348efec0 1 -- 172.21.15.3:0/872714475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18818b0 msgr2=0x1906240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:56.118 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.116+0000 7fc7348efec0 1 --2- 172.21.15.3:0/872714475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18818b0 0x1906240 crc :-1 s=READY pgs=12900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.118 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.116+0000 7fc7348efec0 1 -- 172.21.15.3:0/872714475 shutdown_connections 2024-09-22T04:03:56.118 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.116+0000 7fc7348efec0 1 --2- 172.21.15.3:0/872714475 >> v2:172.21.15.123:6813/891966684 conn(0x191e770 0x19b3990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.118 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.116+0000 7fc7348efec0 1 --2- 172.21.15.3:0/872714475 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc71403d360 0x7fc71403f870 unknown :-1 s=CLOSED pgs=3840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.119 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.116+0000 7fc7348efec0 1 --2- 172.21.15.3:0/872714475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x18818b0 0x1906240 unknown :-1 s=CLOSED pgs=12900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.119 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.116+0000 7fc7348efec0 1 -- 172.21.15.3:0/872714475 >> 172.21.15.3:0/872714475 conn(0x17ae770 msgr2=0x188c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:56.119 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.116+0000 7fc7348efec0 1 -- 172.21.15.3:0/872714475 shutdown_connections 2024-09-22T04:03:56.119 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.116+0000 7fc7348efec0 1 -- 172.21.15.3:0/872714475 wait complete. 2024-09-22T04:03:56.120 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:03:56.120 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p 7c1c124a-aa8e-411d-9731-1f1389755458 put obj9 /etc/passwd 2024-09-22T04:03:56.158 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.157+0000 7f870faebec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:56.164 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.163+0000 7f870faebec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:56.164 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.163+0000 7f870faebec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:56.165 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.164+0000 7f870faebec0 1 Processor -- start 2024-09-22T04:03:56.165 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.164+0000 7f870faebec0 1 -- start start 2024-09-22T04:03:56.167 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.165+0000 7f870faebec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x108a7f0 0x1074690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:56.167 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.165+0000 7f870faebec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0xf5d520 con 0x108a7f0 2024-09-22T04:03:56.167 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.165+0000 7f87037fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x108a7f0 0x1074690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:56.167 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.165+0000 7f87037fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x108a7f0 0x1074690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33322/0 (socket says 172.21.15.3:33322) 2024-09-22T04:03:56.167 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.165+0000 7f87037fe640 1 -- 172.21.15.3:0/1592561400 learned_addr learned my addr 172.21.15.3:0/1592561400 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:56.167 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.166+0000 7f87037fe640 1 -- 172.21.15.3:0/1592561400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xef9e10 con 0x108a7f0 2024-09-22T04:03:56.168 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.166+0000 7f87037fe640 1 --2- 172.21.15.3:0/1592561400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x108a7f0 0x1074690 crc :-1 s=READY pgs=12903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d76567f727a9c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:56.168 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.166+0000 7f8701ffb640 1 -- 172.21.15.3:0/1592561400 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f86f00061f0 con 0x108a7f0 2024-09-22T04:03:56.168 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.166+0000 7f8701ffb640 1 -- 172.21.15.3:0/1592561400 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f86f0001980 con 0x108a7f0 2024-09-22T04:03:56.168 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.166+0000 7f8701ffb640 1 -- 172.21.15.3:0/1592561400 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f86f0001af0 con 0x108a7f0 2024-09-22T04:03:56.169 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.166+0000 7f870faebec0 1 -- 172.21.15.3:0/1592561400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x108a7f0 msgr2=0x1074690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:56.169 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.166+0000 7f870faebec0 1 --2- 172.21.15.3:0/1592561400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x108a7f0 0x1074690 crc :-1 s=READY pgs=12903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.169 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.167+0000 7f870faebec0 1 -- 172.21.15.3:0/1592561400 shutdown_connections 2024-09-22T04:03:56.169 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.167+0000 7f870faebec0 1 --2- 172.21.15.3:0/1592561400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x108a7f0 0x1074690 unknown :-1 s=CLOSED pgs=12903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.169 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.167+0000 7f870faebec0 1 -- 172.21.15.3:0/1592561400 >> 172.21.15.3:0/1592561400 conn(0xf1d770 msgr2=0xfdf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:56.169 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.167+0000 7f870faebec0 1 -- 172.21.15.3:0/1592561400 shutdown_connections 2024-09-22T04:03:56.169 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.167+0000 7f870faebec0 1 -- 172.21.15.3:0/1592561400 wait complete. 2024-09-22T04:03:56.169 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.168+0000 7f870faebec0 1 Processor -- start 2024-09-22T04:03:56.169 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.168+0000 7f870faebec0 1 -- start start 2024-09-22T04:03:56.171 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.169+0000 7f870faebec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x108a7f0 0x10779b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:56.171 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.169+0000 7f870faebec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0xf5b280 con 0x108a7f0 2024-09-22T04:03:56.171 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.169+0000 7f87037fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x108a7f0 0x10779b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:56.171 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.169+0000 7f87037fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x108a7f0 0x10779b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33336/0 (socket says 172.21.15.3:33336) 2024-09-22T04:03:56.171 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.169+0000 7f87037fe640 1 -- 172.21.15.3:0/2686851107 learned_addr learned my addr 172.21.15.3:0/2686851107 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:56.171 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.169+0000 7f87037fe640 1 -- 172.21.15.3:0/2686851107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xeeb920 con 0x108a7f0 2024-09-22T04:03:56.171 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.170+0000 7f87037fe640 1 --2- 172.21.15.3:0/2686851107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x108a7f0 0x10779b0 crc :-1 s=READY pgs=12904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:56.171 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.170+0000 7f8700ff9640 1 -- 172.21.15.3:0/2686851107 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f86f0011d70 con 0x108a7f0 2024-09-22T04:03:56.172 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.170+0000 7f8700ff9640 1 -- 172.21.15.3:0/2686851107 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f86f0012330 con 0x108a7f0 2024-09-22T04:03:56.172 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.170+0000 7f870faebec0 1 -- 172.21.15.3:0/2686851107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0xeeb380 con 0x108a7f0 2024-09-22T04:03:56.172 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.170+0000 7f8700ff9640 1 -- 172.21.15.3:0/2686851107 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f86f00124a0 con 0x108a7f0 2024-09-22T04:03:56.172 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.170+0000 7f870faebec0 1 -- 172.21.15.3:0/2686851107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x107b480 con 0x108a7f0 2024-09-22T04:03:56.174 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.171+0000 7f8700ff9640 1 -- 172.21.15.3:0/2686851107 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f86f00126a0 con 0x108a7f0 2024-09-22T04:03:56.174 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.171+0000 7f8700ff9640 1 --2- 172.21.15.3:0/2686851107 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f86ec03d310 0x7f86ec03f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:56.174 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.171+0000 7f8700ff9640 1 -- 172.21.15.3:0/2686851107 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(707..707 src has 590..707) ==== 3819+0+0 (crc 0 0 0) 0x7f86f004fa70 con 0x108a7f0 2024-09-22T04:03:56.174 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.171+0000 7f870faebec0 1 --2- 172.21.15.3:0/2686851107 >> v2:172.21.15.3:6801/2889521405 conn(0x108b310 0x1522830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:56.174 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.171+0000 7f870faebec0 1 -- 172.21.15.3:0/2686851107 --> v2:172.21.15.3:6801/2889521405 -- osd_op(unknown.0.0:1 199.7 199:fe351e27:::obj9:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e707) -- 0x107aae0 con 0x108b310 2024-09-22T04:03:56.174 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.171+0000 7f8702ffd640 1 --2- 172.21.15.3:0/2686851107 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f86ec03d310 0x7f86ec03f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:56.174 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.172+0000 7f8702ffd640 1 --2- 172.21.15.3:0/2686851107 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f86ec03d310 0x7f86ec03f820 crc :-1 s=READY pgs=3841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:56.175 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.172+0000 7f8703fff640 1 --2- 172.21.15.3:0/2686851107 >> v2:172.21.15.3:6801/2889521405 conn(0x108b310 0x1522830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:03:56.175 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.175+0000 7f8703fff640 1 --2- 172.21.15.3:0/2686851107 >> v2:172.21.15.3:6801/2889521405 conn(0x108b310 0x1522830 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:56.203 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.201+0000 7f8703fff640 1 -- 172.21.15.3:0/2686851107 <== osd.0 v2:172.21.15.3:6801/2889521405 1 ==== osd_op_reply(1 obj9 [writefull 0~12875] v707'2 uv2 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f86f800c430 con 0x108b310 2024-09-22T04:03:56.203 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.201+0000 7f870faebec0 1 -- 172.21.15.3:0/2686851107 >> v2:172.21.15.3:6801/2889521405 conn(0x108b310 msgr2=0x1522830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:56.203 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.201+0000 7f870faebec0 1 --2- 172.21.15.3:0/2686851107 >> v2:172.21.15.3:6801/2889521405 conn(0x108b310 0x1522830 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.203 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.201+0000 7f870faebec0 1 -- 172.21.15.3:0/2686851107 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f86ec03d310 msgr2=0x7f86ec03f820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:56.203 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.201+0000 7f870faebec0 1 --2- 172.21.15.3:0/2686851107 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f86ec03d310 0x7f86ec03f820 crc :-1 s=READY pgs=3841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.204 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.201+0000 7f870faebec0 1 -- 172.21.15.3:0/2686851107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x108a7f0 msgr2=0x10779b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:56.204 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.201+0000 7f870faebec0 1 --2- 172.21.15.3:0/2686851107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x108a7f0 0x10779b0 crc :-1 s=READY pgs=12904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.204 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.202+0000 7f870faebec0 1 -- 172.21.15.3:0/2686851107 shutdown_connections 2024-09-22T04:03:56.204 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.202+0000 7f870faebec0 1 --2- 172.21.15.3:0/2686851107 >> v2:172.21.15.3:6801/2889521405 conn(0x108b310 0x1522830 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.204 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.202+0000 7f870faebec0 1 --2- 172.21.15.3:0/2686851107 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f86ec03d310 0x7f86ec03f820 unknown :-1 s=CLOSED pgs=3841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.204 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.202+0000 7f870faebec0 1 --2- 172.21.15.3:0/2686851107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x108a7f0 0x10779b0 unknown :-1 s=CLOSED pgs=12904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.204 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.202+0000 7f870faebec0 1 -- 172.21.15.3:0/2686851107 >> 172.21.15.3:0/2686851107 conn(0xf1d770 msgr2=0xfdf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:56.204 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.202+0000 7f870faebec0 1 -- 172.21.15.3:0/2686851107 shutdown_connections 2024-09-22T04:03:56.204 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.202+0000 7f870faebec0 1 -- 172.21.15.3:0/2686851107 wait complete. 2024-09-22T04:03:56.205 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:03:56.205 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p 7c1c124a-aa8e-411d-9731-1f1389755458 put obj10 /etc/passwd 2024-09-22T04:03:56.243 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.242+0000 7fe9bf443ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:56.249 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.248+0000 7fe9bf443ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:56.249 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.248+0000 7fe9bf443ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:03:56.250 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.249+0000 7fe9bf443ec0 1 Processor -- start 2024-09-22T04:03:56.250 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.249+0000 7fe9bf443ec0 1 -- start start 2024-09-22T04:03:56.252 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.250+0000 7fe9bf443ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x171cf80 0x183e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:56.252 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.250+0000 7fe9bf443ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x1688520 con 0x171cf80 2024-09-22T04:03:56.252 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.250+0000 7fe9b759b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x171cf80 0x183e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:56.252 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.250+0000 7fe9b759b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x171cf80 0x183e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33346/0 (socket says 172.21.15.3:33346) 2024-09-22T04:03:56.252 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.250+0000 7fe9b759b640 1 -- 172.21.15.3:0/809904659 learned_addr learned my addr 172.21.15.3:0/809904659 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:56.252 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.251+0000 7fe9b759b640 1 -- 172.21.15.3:0/809904659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1624e10 con 0x171cf80 2024-09-22T04:03:56.252 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.251+0000 7fe9b759b640 1 --2- 172.21.15.3:0/809904659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x171cf80 0x183e8a0 crc :-1 s=READY pgs=12907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ca9f0693f50ef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:56.252 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.251+0000 7fe9b5d98640 1 -- 172.21.15.3:0/809904659 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe9a002db90 con 0x171cf80 2024-09-22T04:03:56.253 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.251+0000 7fe9b5d98640 1 -- 172.21.15.3:0/809904659 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe9a0037370 con 0x171cf80 2024-09-22T04:03:56.253 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.251+0000 7fe9b5d98640 1 -- 172.21.15.3:0/809904659 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe9a00374e0 con 0x171cf80 2024-09-22T04:03:56.254 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.251+0000 7fe9bf443ec0 1 -- 172.21.15.3:0/809904659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x171cf80 msgr2=0x183e8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:56.254 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.251+0000 7fe9bf443ec0 1 --2- 172.21.15.3:0/809904659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x171cf80 0x183e8a0 crc :-1 s=READY pgs=12907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.254 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.251+0000 7fe9bf443ec0 1 -- 172.21.15.3:0/809904659 shutdown_connections 2024-09-22T04:03:56.254 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.251+0000 7fe9bf443ec0 1 --2- 172.21.15.3:0/809904659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x171cf80 0x183e8a0 unknown :-1 s=CLOSED pgs=12907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.254 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.251+0000 7fe9bf443ec0 1 -- 172.21.15.3:0/809904659 >> 172.21.15.3:0/809904659 conn(0x1648770 msgr2=0x171f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:56.254 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.251+0000 7fe9bf443ec0 1 -- 172.21.15.3:0/809904659 shutdown_connections 2024-09-22T04:03:56.254 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.252+0000 7fe9bf443ec0 1 -- 172.21.15.3:0/809904659 wait complete. 2024-09-22T04:03:56.254 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.252+0000 7fe9bf443ec0 1 Processor -- start 2024-09-22T04:03:56.254 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.253+0000 7fe9bf443ec0 1 -- start start 2024-09-22T04:03:56.256 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.254+0000 7fe9bf443ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x171cf80 0x184dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:56.256 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.254+0000 7fe9bf443ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x1686280 con 0x171cf80 2024-09-22T04:03:56.256 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.254+0000 7fe9b759b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x171cf80 0x184dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:56.256 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.254+0000 7fe9b759b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x171cf80 0x184dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33352/0 (socket says 172.21.15.3:33352) 2024-09-22T04:03:56.256 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.254+0000 7fe9b759b640 1 -- 172.21.15.3:0/1072227796 learned_addr learned my addr 172.21.15.3:0/1072227796 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:03:56.256 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.254+0000 7fe9b759b640 1 -- 172.21.15.3:0/1072227796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1616920 con 0x171cf80 2024-09-22T04:03:56.256 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.254+0000 7fe9b759b640 1 --2- 172.21.15.3:0/1072227796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x171cf80 0x184dbb0 crc :-1 s=READY pgs=12908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:56.257 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.255+0000 7fe9b4d96640 1 -- 172.21.15.3:0/1072227796 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe9a003e040 con 0x171cf80 2024-09-22T04:03:56.257 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.255+0000 7fe9b4d96640 1 -- 172.21.15.3:0/1072227796 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe9a00057c0 con 0x171cf80 2024-09-22T04:03:56.257 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.255+0000 7fe9bf443ec0 1 -- 172.21.15.3:0/1072227796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1616380 con 0x171cf80 2024-09-22T04:03:56.257 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.255+0000 7fe9b4d96640 1 -- 172.21.15.3:0/1072227796 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe9a002db90 con 0x171cf80 2024-09-22T04:03:56.257 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.255+0000 7fe9bf443ec0 1 -- 172.21.15.3:0/1072227796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x170ba10 con 0x171cf80 2024-09-22T04:03:56.258 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.256+0000 7fe9b4d96640 1 -- 172.21.15.3:0/1072227796 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fe9a0039450 con 0x171cf80 2024-09-22T04:03:56.258 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.256+0000 7fe9b4d96640 1 --2- 172.21.15.3:0/1072227796 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe99c03d360 0x7fe99c03f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:56.258 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.256+0000 7fe9b4d96640 1 -- 172.21.15.3:0/1072227796 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(707..707 src has 590..707) ==== 3819+0+0 (crc 0 0 0) 0x7fe9a0032040 con 0x171cf80 2024-09-22T04:03:56.258 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.256+0000 7fe9bf443ec0 1 --2- 172.21.15.3:0/1072227796 >> v2:172.21.15.3:6801/2889521405 conn(0x17250f0 0x17b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:03:56.258 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.256+0000 7fe9bf443ec0 1 -- 172.21.15.3:0/1072227796 --> v2:172.21.15.3:6801/2889521405 -- osd_op(unknown.0.0:1 199.b 199:d1337354:::obj10:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e707) -- 0x17a8de0 con 0x17250f0 2024-09-22T04:03:56.258 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.256+0000 7fe9b6d9a640 1 --2- 172.21.15.3:0/1072227796 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe99c03d360 0x7fe99c03f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:03:56.258 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.257+0000 7fe9b6d9a640 1 --2- 172.21.15.3:0/1072227796 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe99c03d360 0x7fe99c03f870 crc :-1 s=READY pgs=3842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:56.259 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.257+0000 7fe9b7d9c640 1 --2- 172.21.15.3:0/1072227796 >> v2:172.21.15.3:6801/2889521405 conn(0x17250f0 0x17b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:03:56.262 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.261+0000 7fe9b7d9c640 1 --2- 172.21.15.3:0/1072227796 >> v2:172.21.15.3:6801/2889521405 conn(0x17250f0 0x17b67e0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:03:56.287 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.285+0000 7fe9b7d9c640 1 -- 172.21.15.3:0/1072227796 <== osd.0 v2:172.21.15.3:6801/2889521405 1 ==== osd_op_reply(1 obj10 [writefull 0~12875] v707'1 uv1 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fe9a800a430 con 0x17250f0 2024-09-22T04:03:56.288 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.286+0000 7fe9bf443ec0 1 -- 172.21.15.3:0/1072227796 >> v2:172.21.15.3:6801/2889521405 conn(0x17250f0 msgr2=0x17b67e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:56.288 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.286+0000 7fe9bf443ec0 1 --2- 172.21.15.3:0/1072227796 >> v2:172.21.15.3:6801/2889521405 conn(0x17250f0 0x17b67e0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.288 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.286+0000 7fe9bf443ec0 1 -- 172.21.15.3:0/1072227796 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe99c03d360 msgr2=0x7fe99c03f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:56.288 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.286+0000 7fe9bf443ec0 1 --2- 172.21.15.3:0/1072227796 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe99c03d360 0x7fe99c03f870 crc :-1 s=READY pgs=3842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.288 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.286+0000 7fe9bf443ec0 1 -- 172.21.15.3:0/1072227796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x171cf80 msgr2=0x184dbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:03:56.288 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.286+0000 7fe9bf443ec0 1 --2- 172.21.15.3:0/1072227796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x171cf80 0x184dbb0 crc :-1 s=READY pgs=12908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.288 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.286+0000 7fe9bf443ec0 1 -- 172.21.15.3:0/1072227796 shutdown_connections 2024-09-22T04:03:56.288 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.286+0000 7fe9bf443ec0 1 --2- 172.21.15.3:0/1072227796 >> v2:172.21.15.3:6801/2889521405 conn(0x17250f0 0x17b67e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.288 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.286+0000 7fe9bf443ec0 1 --2- 172.21.15.3:0/1072227796 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe99c03d360 0x7fe99c03f870 unknown :-1 s=CLOSED pgs=3842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.288 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.286+0000 7fe9bf443ec0 1 --2- 172.21.15.3:0/1072227796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x171cf80 0x184dbb0 unknown :-1 s=CLOSED pgs=12908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:03:56.288 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.286+0000 7fe9bf443ec0 1 -- 172.21.15.3:0/1072227796 >> 172.21.15.3:0/1072227796 conn(0x1648770 msgr2=0x1725590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:03:56.288 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.286+0000 7fe9bf443ec0 1 -- 172.21.15.3:0/1072227796 shutdown_connections 2024-09-22T04:03:56.289 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:03:56.286+0000 7fe9bf443ec0 1 -- 172.21.15.3:0/1072227796 wait complete. 2024-09-22T04:03:56.290 INFO:tasks.workunit.client.0.smithi003.stderr:+ sleep 30 2024-09-22T04:04:26.292 INFO:tasks.workunit.client.0.smithi003.stderr:+ pid=95897 2024-09-22T04:04:26.292 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool set-quota 7c1c124a-aa8e-411d-9731-1f1389755458 max_objects 100 2024-09-22T04:04:26.292 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p 7c1c124a-aa8e-411d-9731-1f1389755458 put onemore /etc/passwd 2024-09-22T04:04:26.339 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.337+0000 7f0094774ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:04:26.346 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.344+0000 7f0094774ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:04:26.347 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.344+0000 7f0094774ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:04:26.347 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.346+0000 7f0094774ec0 1 Processor -- start 2024-09-22T04:04:26.348 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.346+0000 7f0094774ec0 1 -- start start 2024-09-22T04:04:26.350 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.347+0000 7f0094774ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1d1cf80 0x1e3e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:04:26.350 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.347+0000 7f0094774ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x1c88520 con 0x1d1cf80 2024-09-22T04:04:26.350 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.347+0000 7f008d19b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1d1cf80 0x1e3e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:04:26.350 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.347+0000 7f008d19b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1d1cf80 0x1e3e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56858/0 (socket says 172.21.15.3:56858) 2024-09-22T04:04:26.350 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.347+0000 7f008d19b640 1 -- 172.21.15.3:0/2717684153 learned_addr learned my addr 172.21.15.3:0/2717684153 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:04:26.350 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.348+0000 7f008d19b640 1 -- 172.21.15.3:0/2717684153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1c24e10 con 0x1d1cf80 2024-09-22T04:04:26.351 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.348+0000 7f008d19b640 1 --2- 172.21.15.3:0/2717684153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1d1cf80 0x1e3e8a0 crc :-1 s=READY pgs=12911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e86b87163541855 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:04:26.351 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.348+0000 7f0086ffd640 1 -- 172.21.15.3:0/2717684153 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f00780061f0 con 0x1d1cf80 2024-09-22T04:04:26.351 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.348+0000 7f0086ffd640 1 -- 172.21.15.3:0/2717684153 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0078001980 con 0x1d1cf80 2024-09-22T04:04:26.351 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.348+0000 7f0086ffd640 1 -- 172.21.15.3:0/2717684153 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0078001af0 con 0x1d1cf80 2024-09-22T04:04:26.352 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.349+0000 7f0094774ec0 1 -- 172.21.15.3:0/2717684153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1d1cf80 msgr2=0x1e3e8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:04:26.352 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.349+0000 7f0094774ec0 1 --2- 172.21.15.3:0/2717684153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1d1cf80 0x1e3e8a0 crc :-1 s=READY pgs=12911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:26.352 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.349+0000 7f0094774ec0 1 -- 172.21.15.3:0/2717684153 shutdown_connections 2024-09-22T04:04:26.352 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.349+0000 7f0094774ec0 1 --2- 172.21.15.3:0/2717684153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1d1cf80 0x1e3e8a0 unknown :-1 s=CLOSED pgs=12911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:26.352 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.350+0000 7f0094774ec0 1 -- 172.21.15.3:0/2717684153 >> 172.21.15.3:0/2717684153 conn(0x1c48770 msgr2=0x1d1e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:04:26.352 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.350+0000 7f0094774ec0 1 -- 172.21.15.3:0/2717684153 shutdown_connections 2024-09-22T04:04:26.352 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.350+0000 7f0094774ec0 1 -- 172.21.15.3:0/2717684153 wait complete. 2024-09-22T04:04:26.353 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.351+0000 7f0094774ec0 1 Processor -- start 2024-09-22T04:04:26.353 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.351+0000 7f0094774ec0 1 -- start start 2024-09-22T04:04:26.356 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.352+0000 7f0094774ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1d1cf80 0x1db6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:04:26.356 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.352+0000 7f0094774ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x1c86280 con 0x1d1cf80 2024-09-22T04:04:26.356 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.353+0000 7f008d19b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1d1cf80 0x1db6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:04:26.356 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.353+0000 7f008d19b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1d1cf80 0x1db6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56866/0 (socket says 172.21.15.3:56866) 2024-09-22T04:04:26.356 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.353+0000 7f008d19b640 1 -- 172.21.15.3:0/3737142173 learned_addr learned my addr 172.21.15.3:0/3737142173 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:04:26.356 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.353+0000 7f008d19b640 1 -- 172.21.15.3:0/3737142173 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1c16920 con 0x1d1cf80 2024-09-22T04:04:26.357 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.353+0000 7f008d19b640 1 --2- 172.21.15.3:0/3737142173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1d1cf80 0x1db6670 crc :-1 s=READY pgs=12912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:04:26.357 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.353+0000 7f0085ffb640 1 -- 172.21.15.3:0/3737142173 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0078011c30 con 0x1d1cf80 2024-09-22T04:04:26.357 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.354+0000 7f0094774ec0 1 -- 172.21.15.3:0/3737142173 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1c16380 con 0x1d1cf80 2024-09-22T04:04:26.357 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.354+0000 7f0085ffb640 1 -- 172.21.15.3:0/3737142173 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f0078011d90 con 0x1d1cf80 2024-09-22T04:04:26.357 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.354+0000 7f0085ffb640 1 -- 172.21.15.3:0/3737142173 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f0078012460 con 0x1d1cf80 2024-09-22T04:04:26.357 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.354+0000 7f0094774ec0 1 -- 172.21.15.3:0/3737142173 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1da45d0 con 0x1d1cf80 2024-09-22T04:04:26.358 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.355+0000 7f0085ffb640 1 -- 172.21.15.3:0/3737142173 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f0078012610 con 0x1d1cf80 2024-09-22T04:04:26.358 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.355+0000 7f0085ffb640 1 --2- 172.21.15.3:0/3737142173 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f007403d360 0x7f007403f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:04:26.358 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.355+0000 7f0085ffb640 1 -- 172.21.15.3:0/3737142173 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(709..709 src has 590..709) ==== 3819+0+0 (crc 0 0 0) 0x7f007804fbb0 con 0x1d1cf80 2024-09-22T04:04:26.358 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.355+0000 7f0085ffb640 1 -- 172.21.15.3:0/3737142173 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=710}) -- 0x7f00740435f0 con 0x1d1cf80 2024-09-22T04:04:26.358 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.355+0000 7f0094774ec0 1 --2- 172.21.15.3:0/3737142173 >> v2:172.21.15.123:6813/891966684 conn(0x1d252c0 0x1da3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:04:26.359 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.356+0000 7f008c99a640 1 --2- 172.21.15.3:0/3737142173 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f007403d360 0x7f007403f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:04:26.359 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.356+0000 7f008c99a640 1 --2- 172.21.15.3:0/3737142173 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f007403d360 0x7f007403f870 crc :-1 s=READY pgs=3843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:04:26.359 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.357+0000 7f008d99c640 1 --2- 172.21.15.3:0/3737142173 >> v2:172.21.15.123:6813/891966684 conn(0x1d252c0 0x1da3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:04:26.363 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.361+0000 7f008d99c640 1 --2- 172.21.15.3:0/3737142173 >> v2:172.21.15.123:6813/891966684 conn(0x1d252c0 0x1da3130 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:04:26.402 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.391+0000 7f57e1d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:04:26.403 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.401+0000 7f57e1d91640 1 Processor -- start 2024-09-22T04:04:26.403 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.401+0000 7f57e1d91640 1 -- start start 2024-09-22T04:04:26.405 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.403+0000 7f57e1d91640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57dc136fe0 0x7f57dc13b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:04:26.405 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.403+0000 7f57e1d91640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57dc0a16a0 con 0x7f57dc1364c0 2024-09-22T04:04:26.405 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.403+0000 7f57e1d91640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f57dc0a0a00 con 0x7f57dc136fe0 2024-09-22T04:04:26.405 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.403+0000 7f57daffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57dc136fe0 0x7f57dc13b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:04:26.405 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.403+0000 7f57db7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f57dc1364c0 0x7f57dc1368d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41766/0 (socket says 172.21.15.3:41766) 2024-09-22T04:04:26.405 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.403+0000 7f57db7fe640 1 -- 172.21.15.3:0/1663188226 learned_addr learned my addr 172.21.15.3:0/1663188226 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T04:04:26.405 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.403+0000 7f57daffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f57dc136fe0 0x7f57dc13b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37978/0 (socket says 172.21.15.3:37978) 2024-09-22T04:04:26.407 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.403+0000 7f57daffd640 1 -- 172.21.15.3:0/1663188226 >> v1:172.21.15.3:6789/0 conn(0x7f57dc1364c0 legacy=0x7f57dc1368d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:04:26.407 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.403+0000 7f57daffd640 1 -- 172.21.15.3:0/1663188226 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc0a13b0 con 0x7f57dc136fe0 2024-09-22T04:04:26.407 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.403+0000 7f57daffd640 1 --2- 172.21.15.3:0/1663188226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57dc136fe0 0x7f57dc13b700 crc :-1 s=READY pgs=12914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4656ad5d2fc602a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:04:26.407 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.404+0000 7f57da7fc640 1 -- 172.21.15.3:0/1663188226 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f57c4009d90 con 0x7f57dc136fe0 2024-09-22T04:04:26.407 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.404+0000 7f57da7fc640 1 -- 172.21.15.3:0/1663188226 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f57c400e230 con 0x7f57dc136fe0 2024-09-22T04:04:26.407 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.404+0000 7f57da7fc640 1 -- 172.21.15.3:0/1663188226 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f57c400e4d0 con 0x7f57dc136fe0 2024-09-22T04:04:26.407 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.404+0000 7f57e1d91640 1 -- 172.21.15.3:0/1663188226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57dc136fe0 msgr2=0x7f57dc13b700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:04:26.407 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.404+0000 7f57e1d91640 1 --2- 172.21.15.3:0/1663188226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57dc136fe0 0x7f57dc13b700 crc :-1 s=READY pgs=12914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:26.407 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.404+0000 7f57e1d91640 1 -- 172.21.15.3:0/1663188226 shutdown_connections 2024-09-22T04:04:26.408 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.404+0000 7f57e1d91640 1 --2- 172.21.15.3:0/1663188226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57dc136fe0 0x7f57dc13b700 unknown :-1 s=CLOSED pgs=12914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:26.408 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.404+0000 7f57e1d91640 1 -- 172.21.15.3:0/1663188226 >> 172.21.15.3:0/1663188226 conn(0x7f57dc0a6270 msgr2=0x7f57dc0a6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:04:26.408 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.405+0000 7f57e1d91640 1 -- 172.21.15.3:0/1663188226 shutdown_connections 2024-09-22T04:04:26.408 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.405+0000 7f57e1d91640 1 -- 172.21.15.3:0/1663188226 wait complete. 2024-09-22T04:04:26.408 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.405+0000 7f57e1d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:04:26.408 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.407+0000 7f57e1d91640 1 Processor -- start 2024-09-22T04:04:26.409 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.407+0000 7f57e1d91640 1 -- start start 2024-09-22T04:04:26.410 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.408+0000 7f57e1d91640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57dc1364c0 0x7f57dc132800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:04:26.411 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.408+0000 7f57e1d91640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc134400 con 0x7f57dc1364c0 2024-09-22T04:04:26.411 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.408+0000 7f57db7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57dc1364c0 0x7f57dc132800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:04:26.411 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.408+0000 7f57db7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57dc1364c0 0x7f57dc132800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37984/0 (socket says 172.21.15.3:37984) 2024-09-22T04:04:26.411 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.408+0000 7f57db7fe640 1 -- 172.21.15.3:0/2289340408 learned_addr learned my addr 172.21.15.3:0/2289340408 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:04:26.412 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.409+0000 7f57db7fe640 1 -- 172.21.15.3:0/2289340408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc0923b0 con 0x7f57dc1364c0 2024-09-22T04:04:26.412 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.409+0000 7f57db7fe640 1 --2- 172.21.15.3:0/2289340408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57dc1364c0 0x7f57dc132800 crc :-1 s=READY pgs=12915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:04:26.412 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.409+0000 7f57d3fff640 1 -- 172.21.15.3:0/2289340408 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f57cc005a60 con 0x7f57dc1364c0 2024-09-22T04:04:26.412 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.409+0000 7f57d3fff640 1 -- 172.21.15.3:0/2289340408 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f57cc005bc0 con 0x7f57dc1364c0 2024-09-22T04:04:26.412 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.409+0000 7f57e1d91640 1 -- 172.21.15.3:0/2289340408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57dc1340f0 con 0x7f57dc1364c0 2024-09-22T04:04:26.412 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.409+0000 7f57d3fff640 1 -- 172.21.15.3:0/2289340408 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f57cc005e60 con 0x7f57dc1364c0 2024-09-22T04:04:26.412 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.409+0000 7f57e1d91640 1 -- 172.21.15.3:0/2289340408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57dc135930 con 0x7f57dc1364c0 2024-09-22T04:04:26.413 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.410+0000 7f57e1d91640 1 -- 172.21.15.3:0/2289340408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57dc0a2230 con 0x7f57dc1364c0 2024-09-22T04:04:26.413 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.411+0000 7f57d3fff640 1 -- 172.21.15.3:0/2289340408 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f57cc00c7f0 con 0x7f57dc1364c0 2024-09-22T04:04:26.413 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.411+0000 7f57d3fff640 1 --2- 172.21.15.3:0/2289340408 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f57bc03ea80 0x7f57bc040f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:04:26.413 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.411+0000 7f57d3fff640 1 -- 172.21.15.3:0/2289340408 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(709..709 src has 590..709) ==== 3819+0+0 (crc 0 0 0) 0x7f57cc04fca0 con 0x7f57dc1364c0 2024-09-22T04:04:26.413 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.411+0000 7f57d3fff640 1 -- 172.21.15.3:0/2289340408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=710}) -- 0x7f57bc044d10 con 0x7f57dc1364c0 2024-09-22T04:04:26.413 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.411+0000 7f57daffd640 1 --2- 172.21.15.3:0/2289340408 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f57bc03ea80 0x7f57bc040f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:04:26.414 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.412+0000 7f57daffd640 1 --2- 172.21.15.3:0/2289340408 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f57bc03ea80 0x7f57bc040f90 crc :-1 s=READY pgs=3844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:04:26.418 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.416+0000 7f57d3fff640 1 -- 172.21.15.3:0/2289340408 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f57cc00dc10 con 0x7f57dc1364c0 2024-09-22T04:04:26.608 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:26.605+0000 7f57e1d91640 1 -- 172.21.15.3:0/2289340408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_objects", "val": "100"} v 0) -- 0x7f57dc050fc0 con 0x7f57dc1364c0 2024-09-22T04:04:27.443 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:27.441+0000 7f57d3fff640 1 -- 172.21.15.3:0/2289340408 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 v710) ==== 225+0+0 (crc 0 0 0) 0x7f57cc00d610 con 0x7f57dc1364c0 2024-09-22T04:04:27.443 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:27.441+0000 7f57d3fff640 1 -- 172.21.15.3:0/2289340408 <== mon.0 v2:172.21.15.3:3300/0 8 ==== osd_map(710..710 src has 590..710) ==== 628+0+0 (crc 0 0 0) 0x7f57cc00d9b0 con 0x7f57dc1364c0 2024-09-22T04:04:27.444 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:27.441+0000 7f0085ffb640 1 -- 172.21.15.3:0/3737142173 <== mon.0 v2:172.21.15.3:3300/0 6 ==== osd_map(710..710 src has 590..710) ==== 628+0+0 (crc 0 0 0) 0x7f0078007040 con 0x1d1cf80 2024-09-22T04:04:27.444 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:27.441+0000 7f0085ffb640 1 -- 172.21.15.3:0/3737142173 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=711}) -- 0x7f0074043d30 con 0x1d1cf80 2024-09-22T04:04:27.444 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:27.441+0000 7f57d3fff640 1 -- 172.21.15.3:0/2289340408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=711}) -- 0x7f57bc045b70 con 0x7f57dc1364c0 2024-09-22T04:04:27.596 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:27.593+0000 7f57e1d91640 1 -- 172.21.15.3:0/2289340408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_objects", "val": "100"} v 0) -- 0x7f57dc074a90 con 0x7f57dc1364c0 2024-09-22T04:04:28.387 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.384+0000 7f57d3fff640 1 -- 172.21.15.3:0/2289340408 <== mon.0 v2:172.21.15.3:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 v711) ==== 225+0+0 (crc 0 0 0) 0x7f57cc00d7f0 con 0x7f57dc1364c0 2024-09-22T04:04:28.387 INFO:tasks.workunit.client.0.smithi003.stderr:set-quota max_objects = 100 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 2024-09-22T04:04:28.387 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.384+0000 7f57d3fff640 1 -- 172.21.15.3:0/2289340408 <== mon.0 v2:172.21.15.3:3300/0 10 ==== osd_map(711..711 src has 590..711) ==== 628+0+0 (crc 0 0 0) 0x7f57cc00de40 con 0x7f57dc1364c0 2024-09-22T04:04:28.387 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.384+0000 7f0085ffb640 1 -- 172.21.15.3:0/3737142173 <== mon.0 v2:172.21.15.3:3300/0 7 ==== osd_map(711..711 src has 590..711) ==== 628+0+0 (crc 0 0 0) 0x7f0078003450 con 0x1d1cf80 2024-09-22T04:04:28.387 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.385+0000 7f57d3fff640 1 -- 172.21.15.3:0/2289340408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=712}) -- 0x7f57bc045d60 con 0x7f57dc1364c0 2024-09-22T04:04:28.387 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.385+0000 7f0085ffb640 1 -- 172.21.15.3:0/3737142173 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=712}) -- 0x7f0074043f50 con 0x1d1cf80 2024-09-22T04:04:28.387 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.385+0000 7f0085ffb640 1 -- 172.21.15.3:0/3737142173 --> v2:172.21.15.123:6813/891966684 -- osd_op(unknown.0.0:1 199.5 199:b9ff7440:::onemore:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e711) -- 0x7f0074044140 con 0x1d252c0 2024-09-22T04:04:28.390 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.387+0000 7f57e1d91640 1 -- 172.21.15.3:0/2289340408 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f57bc03ea80 msgr2=0x7f57bc040f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:04:28.390 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.387+0000 7f57e1d91640 1 --2- 172.21.15.3:0/2289340408 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f57bc03ea80 0x7f57bc040f90 crc :-1 s=READY pgs=3844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.390 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.387+0000 7f57e1d91640 1 -- 172.21.15.3:0/2289340408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57dc1364c0 msgr2=0x7f57dc132800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:04:28.390 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.387+0000 7f57e1d91640 1 --2- 172.21.15.3:0/2289340408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57dc1364c0 0x7f57dc132800 crc :-1 s=READY pgs=12915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.390 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.387+0000 7f57e1d91640 1 -- 172.21.15.3:0/2289340408 shutdown_connections 2024-09-22T04:04:28.390 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.387+0000 7f57e1d91640 1 --2- 172.21.15.3:0/2289340408 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f57bc03ea80 0x7f57bc040f90 unknown :-1 s=CLOSED pgs=3844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.390 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.387+0000 7f57e1d91640 1 --2- 172.21.15.3:0/2289340408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f57dc1364c0 0x7f57dc132800 unknown :-1 s=CLOSED pgs=12915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.390 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.387+0000 7f57e1d91640 1 -- 172.21.15.3:0/2289340408 >> 172.21.15.3:0/2289340408 conn(0x7f57dc0a6270 msgr2=0x7f57dc0716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:04:28.390 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.387+0000 7f57e1d91640 1 -- 172.21.15.3:0/2289340408 shutdown_connections 2024-09-22T04:04:28.390 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.388+0000 7f57e1d91640 1 -- 172.21.15.3:0/2289340408 wait complete. 2024-09-22T04:04:28.408 INFO:tasks.workunit.client.0.smithi003.stderr:+ wait 95897 2024-09-22T04:04:28.460 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.457+0000 7f008d99c640 1 -- 172.21.15.3:0/3737142173 <== osd.2 v2:172.21.15.123:6813/891966684 1 ==== osd_op_reply(1 onemore [writefull 0~12875] v711'2 uv2 ondisk = 0) ==== 151+0+0 (crc 0 0 0) 0x7f008000c430 con 0x1d252c0 2024-09-22T04:04:28.460 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.457+0000 7f0094774ec0 1 -- 172.21.15.3:0/3737142173 >> v2:172.21.15.123:6813/891966684 conn(0x1d252c0 msgr2=0x1da3130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:04:28.460 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.457+0000 7f0094774ec0 1 --2- 172.21.15.3:0/3737142173 >> v2:172.21.15.123:6813/891966684 conn(0x1d252c0 0x1da3130 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.460 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.457+0000 7f0094774ec0 1 -- 172.21.15.3:0/3737142173 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f007403d360 msgr2=0x7f007403f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:04:28.460 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.457+0000 7f0094774ec0 1 --2- 172.21.15.3:0/3737142173 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f007403d360 0x7f007403f870 crc :-1 s=READY pgs=3843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.460 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.457+0000 7f0094774ec0 1 -- 172.21.15.3:0/3737142173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1d1cf80 msgr2=0x1db6670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:04:28.460 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.457+0000 7f0094774ec0 1 --2- 172.21.15.3:0/3737142173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1d1cf80 0x1db6670 crc :-1 s=READY pgs=12912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.461 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.457+0000 7f0094774ec0 1 -- 172.21.15.3:0/3737142173 shutdown_connections 2024-09-22T04:04:28.461 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.457+0000 7f0094774ec0 1 --2- 172.21.15.3:0/3737142173 >> v2:172.21.15.123:6813/891966684 conn(0x1d252c0 0x1da3130 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.461 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.457+0000 7f0094774ec0 1 --2- 172.21.15.3:0/3737142173 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f007403d360 0x7f007403f870 unknown :-1 s=CLOSED pgs=3843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.461 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.457+0000 7f0094774ec0 1 --2- 172.21.15.3:0/3737142173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1d1cf80 0x1db6670 unknown :-1 s=CLOSED pgs=12912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.461 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.457+0000 7f0094774ec0 1 -- 172.21.15.3:0/3737142173 >> 172.21.15.3:0/3737142173 conn(0x1c48770 msgr2=0x1d1e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:04:28.461 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.457+0000 7f0094774ec0 1 -- 172.21.15.3:0/3737142173 shutdown_connections 2024-09-22T04:04:28.461 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.457+0000 7f0094774ec0 1 -- 172.21.15.3:0/3737142173 wait complete. 2024-09-22T04:04:28.462 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -ne 0 ']' 2024-09-22T04:04:28.462 INFO:tasks.workunit.client.0.smithi003.stderr:+ true 2024-09-22T04:04:28.462 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p 7c1c124a-aa8e-411d-9731-1f1389755458 put twomore /etc/passwd 2024-09-22T04:04:28.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.500+0000 7f308def7ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:04:28.507 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.505+0000 7f308def7ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:04:28.508 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.506+0000 7f308def7ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:04:28.509 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.506+0000 7f308def7ec0 1 Processor -- start 2024-09-22T04:04:28.509 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.507+0000 7f308def7ec0 1 -- start start 2024-09-22T04:04:28.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.508+0000 7f308def7ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x8157f0 0x76a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:04:28.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.508+0000 7f308def7ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x6e8520 con 0x8157f0 2024-09-22T04:04:28.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.508+0000 7f308623f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x8157f0 0x76a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:04:28.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.508+0000 7f308623f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x8157f0 0x76a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38006/0 (socket says 172.21.15.3:38006) 2024-09-22T04:04:28.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.508+0000 7f308623f640 1 -- 172.21.15.3:0/3465964293 learned_addr learned my addr 172.21.15.3:0/3465964293 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:04:28.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.508+0000 7f308623f640 1 -- 172.21.15.3:0/3465964293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x684e10 con 0x8157f0 2024-09-22T04:04:28.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.508+0000 7f308623f640 1 --2- 172.21.15.3:0/3465964293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x8157f0 0x76a400 crc :-1 s=READY pgs=12918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e5138b8b3ac35 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:04:28.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.509+0000 7f3084a3c640 1 -- 172.21.15.3:0/3465964293 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f307800b580 con 0x8157f0 2024-09-22T04:04:28.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.509+0000 7f3084a3c640 1 -- 172.21.15.3:0/3465964293 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f307800b6e0 con 0x8157f0 2024-09-22T04:04:28.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.509+0000 7f3084a3c640 1 -- 172.21.15.3:0/3465964293 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f307800b980 con 0x8157f0 2024-09-22T04:04:28.512 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.509+0000 7f308def7ec0 1 -- 172.21.15.3:0/3465964293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x8157f0 msgr2=0x76a400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:04:28.513 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.509+0000 7f308def7ec0 1 --2- 172.21.15.3:0/3465964293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x8157f0 0x76a400 crc :-1 s=READY pgs=12918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.513 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.509+0000 7f308def7ec0 1 -- 172.21.15.3:0/3465964293 shutdown_connections 2024-09-22T04:04:28.513 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.509+0000 7f308def7ec0 1 --2- 172.21.15.3:0/3465964293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x8157f0 0x76a400 unknown :-1 s=CLOSED pgs=12918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.513 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.509+0000 7f308def7ec0 1 -- 172.21.15.3:0/3465964293 >> 172.21.15.3:0/3465964293 conn(0x6a8770 msgr2=0x77d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:04:28.513 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.509+0000 7f308def7ec0 1 -- 172.21.15.3:0/3465964293 shutdown_connections 2024-09-22T04:04:28.513 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.509+0000 7f308def7ec0 1 -- 172.21.15.3:0/3465964293 wait complete. 2024-09-22T04:04:28.513 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.510+0000 7f308def7ec0 1 Processor -- start 2024-09-22T04:04:28.513 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.510+0000 7f308def7ec0 1 -- start start 2024-09-22T04:04:28.515 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.512+0000 7f308def7ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x8157f0 0x807370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:04:28.515 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.512+0000 7f308def7ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x6e6280 con 0x8157f0 2024-09-22T04:04:28.515 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.512+0000 7f308623f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x8157f0 0x807370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:04:28.515 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.512+0000 7f308623f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x8157f0 0x807370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38008/0 (socket says 172.21.15.3:38008) 2024-09-22T04:04:28.515 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.512+0000 7f308623f640 1 -- 172.21.15.3:0/561527639 learned_addr learned my addr 172.21.15.3:0/561527639 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:04:28.515 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.512+0000 7f308623f640 1 -- 172.21.15.3:0/561527639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x676920 con 0x8157f0 2024-09-22T04:04:28.515 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.512+0000 7f308623f640 1 --2- 172.21.15.3:0/561527639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x8157f0 0x807370 crc :-1 s=READY pgs=12919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:04:28.515 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.513+0000 7f307f7fe640 1 -- 172.21.15.3:0/561527639 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3078012b10 con 0x8157f0 2024-09-22T04:04:28.515 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.513+0000 7f307f7fe640 1 -- 172.21.15.3:0/561527639 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3078005040 con 0x8157f0 2024-09-22T04:04:28.515 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.513+0000 7f308def7ec0 1 -- 172.21.15.3:0/561527639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x676380 con 0x8157f0 2024-09-22T04:04:28.516 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.513+0000 7f307f7fe640 1 -- 172.21.15.3:0/561527639 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3078012da0 con 0x8157f0 2024-09-22T04:04:28.516 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.513+0000 7f308def7ec0 1 -- 172.21.15.3:0/561527639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x80b1e0 con 0x8157f0 2024-09-22T04:04:28.516 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.514+0000 7f307f7fe640 1 -- 172.21.15.3:0/561527639 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f307800d040 con 0x8157f0 2024-09-22T04:04:28.516 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.514+0000 7f307f7fe640 1 --2- 172.21.15.3:0/561527639 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f306003d310 0x7f306003f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:04:28.517 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.514+0000 7f307f7fe640 1 -- 172.21.15.3:0/561527639 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(711..711 src has 590..711) ==== 3819+0+0 (crc 0 0 0) 0x7f307800a040 con 0x8157f0 2024-09-22T04:04:28.517 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.514+0000 7f308def7ec0 1 --2- 172.21.15.3:0/561527639 >> v2:172.21.15.123:6806/4206675901 conn(0x816310 0x8ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:04:28.517 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.514+0000 7f308def7ec0 1 -- 172.21.15.3:0/561527639 --> v2:172.21.15.123:6806/4206675901 -- osd_op(unknown.0.0:1 199.6 199:7019d845:::twomore:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e711) -- 0x80ad50 con 0x816310 2024-09-22T04:04:28.517 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.514+0000 7f3085a3e640 1 --2- 172.21.15.3:0/561527639 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f306003d310 0x7f306003f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:04:28.517 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.515+0000 7f3085a3e640 1 --2- 172.21.15.3:0/561527639 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f306003d310 0x7f306003f820 crc :-1 s=READY pgs=3845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:04:28.517 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.515+0000 7f3086a40640 1 --2- 172.21.15.3:0/561527639 >> v2:172.21.15.123:6806/4206675901 conn(0x816310 0x8ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:04:28.520 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.518+0000 7f3086a40640 1 --2- 172.21.15.3:0/561527639 >> v2:172.21.15.123:6806/4206675901 conn(0x816310 0x8ad840 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:04:28.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.542+0000 7f3086a40640 1 -- 172.21.15.3:0/561527639 <== osd.3 v2:172.21.15.123:6806/4206675901 1 ==== osd_op_reply(1 twomore [writefull 0~12875] v711'1 uv1 ondisk = 0) ==== 151+0+0 (crc 0 0 0) 0x7f307000a430 con 0x816310 2024-09-22T04:04:28.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.543+0000 7f308def7ec0 1 -- 172.21.15.3:0/561527639 >> v2:172.21.15.123:6806/4206675901 conn(0x816310 msgr2=0x8ad840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:04:28.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.543+0000 7f308def7ec0 1 --2- 172.21.15.3:0/561527639 >> v2:172.21.15.123:6806/4206675901 conn(0x816310 0x8ad840 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.543+0000 7f308def7ec0 1 -- 172.21.15.3:0/561527639 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f306003d310 msgr2=0x7f306003f820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:04:28.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.543+0000 7f308def7ec0 1 --2- 172.21.15.3:0/561527639 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f306003d310 0x7f306003f820 crc :-1 s=READY pgs=3845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.543+0000 7f308def7ec0 1 -- 172.21.15.3:0/561527639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x8157f0 msgr2=0x807370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:04:28.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.543+0000 7f308def7ec0 1 --2- 172.21.15.3:0/561527639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x8157f0 0x807370 crc :-1 s=READY pgs=12919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.543+0000 7f308def7ec0 1 -- 172.21.15.3:0/561527639 shutdown_connections 2024-09-22T04:04:28.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.543+0000 7f308def7ec0 1 --2- 172.21.15.3:0/561527639 >> v2:172.21.15.123:6806/4206675901 conn(0x816310 0x8ad840 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.543+0000 7f308def7ec0 1 --2- 172.21.15.3:0/561527639 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f306003d310 0x7f306003f820 unknown :-1 s=CLOSED pgs=3845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.543+0000 7f308def7ec0 1 --2- 172.21.15.3:0/561527639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x8157f0 0x807370 unknown :-1 s=CLOSED pgs=12919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.543+0000 7f308def7ec0 1 -- 172.21.15.3:0/561527639 >> 172.21.15.3:0/561527639 conn(0x6a8770 msgr2=0x77d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:04:28.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.543+0000 7f308def7ec0 1 -- 172.21.15.3:0/561527639 shutdown_connections 2024-09-22T04:04:28.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.543+0000 7f308def7ec0 1 -- 172.21.15.3:0/561527639 wait complete. 2024-09-22T04:04:28.548 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool set-quota 7c1c124a-aa8e-411d-9731-1f1389755458 max_bytes 100 2024-09-22T04:04:28.644 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.633+0000 7f853cf08640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:04:28.645 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.643+0000 7f853cf08640 1 Processor -- start 2024-09-22T04:04:28.645 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.643+0000 7f853cf08640 1 -- start start 2024-09-22T04:04:28.647 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.645+0000 7f853cf08640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f85380a0390 0x7f853813b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:04:28.647 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.645+0000 7f853cf08640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8538074b50 con 0x7f853809f870 2024-09-22T04:04:28.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.645+0000 7f853cf08640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f85380a0c60 con 0x7f85380a0390 2024-09-22T04:04:28.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.645+0000 7f8535d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f85380a0390 0x7f853813b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:04:28.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.645+0000 7f853659c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f853809f870 0x7f853809fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41772/0 (socket says 172.21.15.3:41772) 2024-09-22T04:04:28.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.645+0000 7f8535d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f85380a0390 0x7f853813b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38020/0 (socket says 172.21.15.3:38020) 2024-09-22T04:04:28.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.645+0000 7f853659c640 1 -- 172.21.15.3:0/1450070367 learned_addr learned my addr 172.21.15.3:0/1450070367 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T04:04:28.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.645+0000 7f8535d9b640 1 -- 172.21.15.3:0/1450070367 >> v1:172.21.15.3:6789/0 conn(0x7f853809f870 legacy=0x7f853809fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:04:28.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.645+0000 7f853559a640 1 -- 172.21.15.3:0/1450070367 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4042119668 0 0) 0x7f8538074b50 con 0x7f853809f870 2024-09-22T04:04:28.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.645+0000 7f8535d9b640 1 -- 172.21.15.3:0/1450070367 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85380a2110 con 0x7f85380a0390 2024-09-22T04:04:28.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.645+0000 7f8535d9b640 1 --2- 172.21.15.3:0/1450070367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85380a0390 0x7f853813b700 crc :-1 s=READY pgs=12921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c7c581f3d2ab92 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:04:28.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.645+0000 7f853559a640 1 -- 172.21.15.3:0/1450070367 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f852c009d80 con 0x7f85380a0390 2024-09-22T04:04:28.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.645+0000 7f853559a640 1 -- 172.21.15.3:0/1450070367 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f852c00e230 con 0x7f85380a0390 2024-09-22T04:04:28.649 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.645+0000 7f853559a640 1 -- 172.21.15.3:0/1450070367 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f852c00e4d0 con 0x7f85380a0390 2024-09-22T04:04:28.650 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.646+0000 7f853cf08640 1 -- 172.21.15.3:0/1450070367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85380a0390 msgr2=0x7f853813b700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:04:28.650 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.646+0000 7f853cf08640 1 --2- 172.21.15.3:0/1450070367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85380a0390 0x7f853813b700 crc :-1 s=READY pgs=12921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.650 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.646+0000 7f853cf08640 1 -- 172.21.15.3:0/1450070367 shutdown_connections 2024-09-22T04:04:28.650 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.646+0000 7f853cf08640 1 --2- 172.21.15.3:0/1450070367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85380a0390 0x7f853813b700 unknown :-1 s=CLOSED pgs=12921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:28.650 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.646+0000 7f853cf08640 1 -- 172.21.15.3:0/1450070367 >> 172.21.15.3:0/1450070367 conn(0x7f85380a6270 msgr2=0x7f85380a6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:04:28.650 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.646+0000 7f853cf08640 1 -- 172.21.15.3:0/1450070367 shutdown_connections 2024-09-22T04:04:28.650 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.646+0000 7f853cf08640 1 -- 172.21.15.3:0/1450070367 wait complete. 2024-09-22T04:04:28.650 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.646+0000 7f853cf08640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:04:28.651 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.648+0000 7f853cf08640 1 Processor -- start 2024-09-22T04:04:28.651 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.648+0000 7f853cf08640 1 -- start start 2024-09-22T04:04:28.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.650+0000 7f853cf08640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f853809f870 0x7f85381ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:04:28.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.650+0000 7f853cf08640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f85380a2430 con 0x7f853809f870 2024-09-22T04:04:28.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.650+0000 7f853659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f853809f870 0x7f85381ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:04:28.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.650+0000 7f853659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f853809f870 0x7f85381ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38036/0 (socket says 172.21.15.3:38036) 2024-09-22T04:04:28.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.650+0000 7f853659c640 1 -- 172.21.15.3:0/706249700 learned_addr learned my addr 172.21.15.3:0/706249700 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:04:28.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.650+0000 7f853659c640 1 -- 172.21.15.3:0/706249700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8538137420 con 0x7f853809f870 2024-09-22T04:04:28.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.650+0000 7f853659c640 1 --2- 172.21.15.3:0/706249700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f853809f870 0x7f85381ca850 crc :-1 s=READY pgs=12922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:04:28.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.650+0000 7f8526ffd640 1 -- 172.21.15.3:0/706249700 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8528004b80 con 0x7f853809f870 2024-09-22T04:04:28.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.651+0000 7f853cf08640 1 -- 172.21.15.3:0/706249700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8538133430 con 0x7f853809f870 2024-09-22T04:04:28.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.651+0000 7f8526ffd640 1 -- 172.21.15.3:0/706249700 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8528004ce0 con 0x7f853809f870 2024-09-22T04:04:28.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.651+0000 7f8526ffd640 1 -- 172.21.15.3:0/706249700 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8528007120 con 0x7f853809f870 2024-09-22T04:04:28.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.651+0000 7f853cf08640 1 -- 172.21.15.3:0/706249700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8538137110 con 0x7f853809f870 2024-09-22T04:04:28.654 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.652+0000 7f853cf08640 1 -- 172.21.15.3:0/706249700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f853809fc80 con 0x7f853809f870 2024-09-22T04:04:28.654 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.652+0000 7f8526ffd640 1 -- 172.21.15.3:0/706249700 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f8528007320 con 0x7f853809f870 2024-09-22T04:04:28.654 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.652+0000 7f8526ffd640 1 --2- 172.21.15.3:0/706249700 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f851803ea80 0x7f8518040f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:04:28.654 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.652+0000 7f8526ffd640 1 -- 172.21.15.3:0/706249700 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(711..711 src has 590..711) ==== 3819+0+0 (crc 0 0 0) 0x7f852804fe60 con 0x7f853809f870 2024-09-22T04:04:28.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.652+0000 7f8535d9b640 1 --2- 172.21.15.3:0/706249700 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f851803ea80 0x7f8518040f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:04:28.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.653+0000 7f8535d9b640 1 --2- 172.21.15.3:0/706249700 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f851803ea80 0x7f8518040f90 crc :-1 s=READY pgs=3846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:04:28.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.657+0000 7f8526ffd640 1 -- 172.21.15.3:0/706249700 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f852800f230 con 0x7f853809f870 2024-09-22T04:04:28.851 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:28.848+0000 7f853cf08640 1 -- 172.21.15.3:0/706249700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_bytes", "val": "100"} v 0) -- 0x7f8538050fc0 con 0x7f853809f870 2024-09-22T04:04:29.452 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:29.449+0000 7f8526ffd640 1 -- 172.21.15.3:0/706249700 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 v712) ==== 221+0+0 (crc 0 0 0) 0x7f852800a0c0 con 0x7f853809f870 2024-09-22T04:04:29.611 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:29.609+0000 7f853cf08640 1 -- 172.21.15.3:0/706249700 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_bytes", "val": "100"} v 0) -- 0x7f85380a1130 con 0x7f853809f870 2024-09-22T04:04:30.456 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:30.453+0000 7f8526ffd640 1 -- 172.21.15.3:0/706249700 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 v713) ==== 221+0+0 (crc 0 0 0) 0x7f852800a7b0 con 0x7f853809f870 2024-09-22T04:04:30.456 INFO:tasks.workunit.client.0.smithi003.stderr:set-quota max_bytes = 100 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 2024-09-22T04:04:30.461 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:30.458+0000 7f853cf08640 1 -- 172.21.15.3:0/706249700 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f851803ea80 msgr2=0x7f8518040f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:04:30.461 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:30.458+0000 7f853cf08640 1 --2- 172.21.15.3:0/706249700 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f851803ea80 0x7f8518040f90 crc :-1 s=READY pgs=3846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:30.461 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:30.458+0000 7f853cf08640 1 -- 172.21.15.3:0/706249700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f853809f870 msgr2=0x7f85381ca850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:04:30.461 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:30.458+0000 7f853cf08640 1 --2- 172.21.15.3:0/706249700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f853809f870 0x7f85381ca850 crc :-1 s=READY pgs=12922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:30.461 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:30.458+0000 7f853cf08640 1 -- 172.21.15.3:0/706249700 shutdown_connections 2024-09-22T04:04:30.461 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:30.458+0000 7f853cf08640 1 --2- 172.21.15.3:0/706249700 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f851803ea80 0x7f8518040f90 unknown :-1 s=CLOSED pgs=3846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:30.461 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:30.458+0000 7f853cf08640 1 --2- 172.21.15.3:0/706249700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f853809f870 0x7f85381ca850 unknown :-1 s=CLOSED pgs=12922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:04:30.461 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:30.458+0000 7f853cf08640 1 -- 172.21.15.3:0/706249700 >> 172.21.15.3:0/706249700 conn(0x7f85380a6270 msgr2=0x7f85380a3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:04:30.461 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:30.458+0000 7f853cf08640 1 -- 172.21.15.3:0/706249700 shutdown_connections 2024-09-22T04:04:30.461 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:04:30.458+0000 7f853cf08640 1 -- 172.21.15.3:0/706249700 wait complete. 2024-09-22T04:04:30.479 INFO:tasks.workunit.client.0.smithi003.stderr:+ sleep 30 2024-09-22T04:05:00.481 INFO:tasks.workunit.client.0.smithi003.stderr:+ pid=95989 2024-09-22T04:05:00.481 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool set-quota 7c1c124a-aa8e-411d-9731-1f1389755458 max_bytes 0 2024-09-22T04:05:00.481 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p 7c1c124a-aa8e-411d-9731-1f1389755458 put two /etc/passwd 2024-09-22T04:05:00.528 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.526+0000 7f4d20c76ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:00.535 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.534+0000 7f4d20c76ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:00.535 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.534+0000 7f4d20c76ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:00.536 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.535+0000 7f4d20c76ec0 1 Processor -- start 2024-09-22T04:05:00.536 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.535+0000 7f4d20c76ec0 1 -- start start 2024-09-22T04:05:00.538 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.537+0000 7f4d20c76ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16bd9c0 0x1751180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:00.538 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.537+0000 7f4d20c76ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x1622520 con 0x16bd9c0 2024-09-22T04:05:00.539 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.537+0000 7f4d18d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16bd9c0 0x1751180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:00.539 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.537+0000 7f4d18d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16bd9c0 0x1751180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36076/0 (socket says 172.21.15.3:36076) 2024-09-22T04:05:00.539 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.537+0000 7f4d18d9b640 1 -- 172.21.15.3:0/257994511 learned_addr learned my addr 172.21.15.3:0/257994511 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:00.539 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.538+0000 7f4d18d9b640 1 -- 172.21.15.3:0/257994511 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x15bee10 con 0x16bd9c0 2024-09-22T04:05:00.539 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.538+0000 7f4d18d9b640 1 --2- 172.21.15.3:0/257994511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16bd9c0 0x1751180 crc :-1 s=READY pgs=12925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595918d3eb770b87 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:00.539 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.538+0000 7f4d127fc640 1 -- 172.21.15.3:0/257994511 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4cfc00b580 con 0x16bd9c0 2024-09-22T04:05:00.540 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.538+0000 7f4d127fc640 1 -- 172.21.15.3:0/257994511 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4cfc00b6e0 con 0x16bd9c0 2024-09-22T04:05:00.540 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.538+0000 7f4d127fc640 1 -- 172.21.15.3:0/257994511 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4cfc00b980 con 0x16bd9c0 2024-09-22T04:05:00.540 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.539+0000 7f4d20c76ec0 1 -- 172.21.15.3:0/257994511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16bd9c0 msgr2=0x1751180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:00.540 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.539+0000 7f4d20c76ec0 1 --2- 172.21.15.3:0/257994511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16bd9c0 0x1751180 crc :-1 s=READY pgs=12925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:00.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.539+0000 7f4d20c76ec0 1 -- 172.21.15.3:0/257994511 shutdown_connections 2024-09-22T04:05:00.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.539+0000 7f4d20c76ec0 1 --2- 172.21.15.3:0/257994511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16bd9c0 0x1751180 unknown :-1 s=CLOSED pgs=12925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:00.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.539+0000 7f4d20c76ec0 1 -- 172.21.15.3:0/257994511 >> 172.21.15.3:0/257994511 conn(0x15e2770 msgr2=0x16bad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:00.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.539+0000 7f4d20c76ec0 1 -- 172.21.15.3:0/257994511 shutdown_connections 2024-09-22T04:05:00.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.539+0000 7f4d20c76ec0 1 -- 172.21.15.3:0/257994511 wait complete. 2024-09-22T04:05:00.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.540+0000 7f4d20c76ec0 1 Processor -- start 2024-09-22T04:05:00.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.540+0000 7f4d20c76ec0 1 -- start start 2024-09-22T04:05:00.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.542+0000 7f4d20c76ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16bd9c0 0x173b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:00.544 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.542+0000 7f4d20c76ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x1620280 con 0x16bd9c0 2024-09-22T04:05:00.544 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.542+0000 7f4d18d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16bd9c0 0x173b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:00.544 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.542+0000 7f4d18d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16bd9c0 0x173b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36086/0 (socket says 172.21.15.3:36086) 2024-09-22T04:05:00.544 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.542+0000 7f4d18d9b640 1 -- 172.21.15.3:0/1687569659 learned_addr learned my addr 172.21.15.3:0/1687569659 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:00.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.543+0000 7f4d18d9b640 1 -- 172.21.15.3:0/1687569659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x15b0920 con 0x16bd9c0 2024-09-22T04:05:00.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.543+0000 7f4d18d9b640 1 --2- 172.21.15.3:0/1687569659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16bd9c0 0x173b9c0 crc :-1 s=READY pgs=12926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:00.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.543+0000 7f4d117fa640 1 -- 172.21.15.3:0/1687569659 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4cfc012b10 con 0x16bd9c0 2024-09-22T04:05:00.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.543+0000 7f4d117fa640 1 -- 172.21.15.3:0/1687569659 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4cfc005040 con 0x16bd9c0 2024-09-22T04:05:00.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.543+0000 7f4d20c76ec0 1 -- 172.21.15.3:0/1687569659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x15b0380 con 0x16bd9c0 2024-09-22T04:05:00.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.543+0000 7f4d117fa640 1 -- 172.21.15.3:0/1687569659 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4cfc012da0 con 0x16bd9c0 2024-09-22T04:05:00.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.543+0000 7f4d20c76ec0 1 -- 172.21.15.3:0/1687569659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x173e920 con 0x16bd9c0 2024-09-22T04:05:00.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.544+0000 7f4d117fa640 1 -- 172.21.15.3:0/1687569659 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f4cfc00d040 con 0x16bd9c0 2024-09-22T04:05:00.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.545+0000 7f4d117fa640 1 --2- 172.21.15.3:0/1687569659 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4cf403d360 0x7f4cf403f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:00.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.545+0000 7f4d117fa640 1 -- 172.21.15.3:0/1687569659 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(715..715 src has 590..715) ==== 3819+0+0 (crc 0 0 0) 0x7f4cfc00a040 con 0x16bd9c0 2024-09-22T04:05:00.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.545+0000 7f4d117fa640 1 -- 172.21.15.3:0/1687569659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=716}) -- 0x7f4cf4043650 con 0x16bd9c0 2024-09-22T04:05:00.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.545+0000 7f4d20c76ec0 1 --2- 172.21.15.3:0/1687569659 >> v2:172.21.15.123:6806/4206675901 conn(0x16be4e0 0x1be9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:00.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.545+0000 7f4d137fe640 1 --2- 172.21.15.3:0/1687569659 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4cf403d360 0x7f4cf403f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:00.547 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.546+0000 7f4d137fe640 1 --2- 172.21.15.3:0/1687569659 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4cf403d360 0x7f4cf403f870 crc :-1 s=READY pgs=3847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:00.547 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.546+0000 7f4d1959c640 1 --2- 172.21.15.3:0/1687569659 >> v2:172.21.15.123:6806/4206675901 conn(0x16be4e0 0x1be9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:05:00.552 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.551+0000 7f4d1959c640 1 --2- 172.21.15.3:0/1687569659 >> v2:172.21.15.123:6806/4206675901 conn(0x16be4e0 0x1be9840 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:00.592 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.582+0000 7f361cb51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:00.593 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.592+0000 7f361cb51640 1 Processor -- start 2024-09-22T04:05:00.593 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.592+0000 7f361cb51640 1 -- start start 2024-09-22T04:05:00.597 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.594+0000 7f361cb51640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3618133d80 0x7f3618134190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:00.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.594+0000 7f361cb51640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36180748f0 con 0x7f361809da20 2024-09-22T04:05:00.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.594+0000 7f361cb51640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f36180a1110 con 0x7f3618133d80 2024-09-22T04:05:00.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.594+0000 7f361659c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3618133d80 0x7f3618134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:00.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.594+0000 7f361659c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3618133d80 0x7f3618134190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36096/0 (socket says 172.21.15.3:36096) 2024-09-22T04:05:00.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.594+0000 7f3615d9b640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f361809da20 0x7f361809de30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51612/0 (socket says 172.21.15.3:51612) 2024-09-22T04:05:00.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.594+0000 7f361659c640 1 -- 172.21.15.3:0/919188944 learned_addr learned my addr 172.21.15.3:0/919188944 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:00.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.594+0000 7f361659c640 1 -- 172.21.15.3:0/919188944 >> v1:172.21.15.3:6789/0 conn(0x7f361809da20 legacy=0x7f361809de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:00.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.594+0000 7f361559a640 1 -- 172.21.15.3:0/919188944 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1298218530 0 0) 0x7f36180748f0 con 0x7f361809da20 2024-09-22T04:05:00.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.594+0000 7f361659c640 1 -- 172.21.15.3:0/919188944 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361809f5b0 con 0x7f3618133d80 2024-09-22T04:05:00.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.594+0000 7f361659c640 1 --2- 172.21.15.3:0/919188944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3618133d80 0x7f3618134190 crc :-1 s=READY pgs=12928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b65cdb9a3b413ef server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:00.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.595+0000 7f361559a640 1 -- 172.21.15.3:0/919188944 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f360800e040 con 0x7f3618133d80 2024-09-22T04:05:00.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.595+0000 7f361559a640 1 -- 172.21.15.3:0/919188944 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f36080091e0 con 0x7f3618133d80 2024-09-22T04:05:00.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.595+0000 7f361559a640 1 -- 172.21.15.3:0/919188944 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3608007730 con 0x7f3618133d80 2024-09-22T04:05:00.599 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.595+0000 7f361cb51640 1 -- 172.21.15.3:0/919188944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3618133d80 msgr2=0x7f3618134190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:00.599 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.595+0000 7f361cb51640 1 --2- 172.21.15.3:0/919188944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3618133d80 0x7f3618134190 crc :-1 s=READY pgs=12928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:00.600 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.595+0000 7f361cb51640 1 -- 172.21.15.3:0/919188944 shutdown_connections 2024-09-22T04:05:00.600 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.595+0000 7f361cb51640 1 --2- 172.21.15.3:0/919188944 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3618133d80 0x7f3618134190 unknown :-1 s=CLOSED pgs=12928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:00.600 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.595+0000 7f361cb51640 1 -- 172.21.15.3:0/919188944 >> 172.21.15.3:0/919188944 conn(0x7f36180a6d10 msgr2=0x7f3618073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:00.600 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.595+0000 7f361cb51640 1 -- 172.21.15.3:0/919188944 shutdown_connections 2024-09-22T04:05:00.600 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.596+0000 7f361cb51640 1 -- 172.21.15.3:0/919188944 wait complete. 2024-09-22T04:05:00.600 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.596+0000 7f361cb51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:00.600 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.598+0000 7f361cb51640 1 Processor -- start 2024-09-22T04:05:00.600 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.598+0000 7f361cb51640 1 -- start start 2024-09-22T04:05:00.601 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.599+0000 7f361cb51640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f361809da20 0x7f36181208a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:00.601 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.599+0000 7f361cb51640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f36181338c0 con 0x7f361809da20 2024-09-22T04:05:00.602 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.599+0000 7f361659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f361809da20 0x7f36181208a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:00.602 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.599+0000 7f361659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f361809da20 0x7f36181208a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36102/0 (socket says 172.21.15.3:36102) 2024-09-22T04:05:00.602 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.599+0000 7f361659c640 1 -- 172.21.15.3:0/1407891186 learned_addr learned my addr 172.21.15.3:0/1407891186 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:00.602 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.600+0000 7f361659c640 1 -- 172.21.15.3:0/1407891186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3618123ed0 con 0x7f361809da20 2024-09-22T04:05:00.602 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.600+0000 7f361659c640 1 --2- 172.21.15.3:0/1407891186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f361809da20 0x7f36181208a0 crc :-1 s=READY pgs=12929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:00.602 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.600+0000 7f3606ffd640 1 -- 172.21.15.3:0/1407891186 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f360800e040 con 0x7f361809da20 2024-09-22T04:05:00.602 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.600+0000 7f3606ffd640 1 -- 172.21.15.3:0/1407891186 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3608012790 con 0x7f361809da20 2024-09-22T04:05:00.602 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.600+0000 7f361cb51640 1 -- 172.21.15.3:0/1407891186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3618122140 con 0x7f361809da20 2024-09-22T04:05:00.602 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.600+0000 7f3606ffd640 1 -- 172.21.15.3:0/1407891186 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3608012a30 con 0x7f361809da20 2024-09-22T04:05:00.602 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.600+0000 7f361cb51640 1 -- 172.21.15.3:0/1407891186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36181245f0 con 0x7f361809da20 2024-09-22T04:05:00.603 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.601+0000 7f361cb51640 1 -- 172.21.15.3:0/1407891186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36180a5cc0 con 0x7f361809da20 2024-09-22T04:05:00.603 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.601+0000 7f3606ffd640 1 -- 172.21.15.3:0/1407891186 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f3608012be0 con 0x7f361809da20 2024-09-22T04:05:00.603 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.602+0000 7f3606ffd640 1 --2- 172.21.15.3:0/1407891186 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f35f803ead0 0x7f35f8040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:00.603 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.602+0000 7f3606ffd640 1 -- 172.21.15.3:0/1407891186 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(715..715 src has 590..715) ==== 3819+0+0 (crc 0 0 0) 0x7f36080512c0 con 0x7f361809da20 2024-09-22T04:05:00.603 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.602+0000 7f3606ffd640 1 -- 172.21.15.3:0/1407891186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=716}) -- 0x7f35f8044dc0 con 0x7f361809da20 2024-09-22T04:05:00.603 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.602+0000 7f3615d9b640 1 --2- 172.21.15.3:0/1407891186 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f35f803ead0 0x7f35f8040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:00.604 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.603+0000 7f3615d9b640 1 --2- 172.21.15.3:0/1407891186 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f35f803ead0 0x7f35f8040fe0 crc :-1 s=READY pgs=3848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:00.608 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.607+0000 7f3606ffd640 1 -- 172.21.15.3:0/1407891186 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f3608014570 con 0x7f361809da20 2024-09-22T04:05:00.800 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.799+0000 7f361cb51640 1 -- 172.21.15.3:0/1407891186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_bytes", "val": "0"} v 0) -- 0x7f3618050fc0 con 0x7f361809da20 2024-09-22T04:05:00.864 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.863+0000 7f3606ffd640 1 -- 172.21.15.3:0/1407891186 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 v716) ==== 217+0+0 (crc 0 0 0) 0x7f3608004860 con 0x7f361809da20 2024-09-22T04:05:00.866 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.864+0000 7f3606ffd640 1 -- 172.21.15.3:0/1407891186 <== mon.0 v2:172.21.15.3:3300/0 8 ==== osd_map(716..716 src has 590..716) ==== 628+0+0 (crc 0 0 0) 0x7f360800e500 con 0x7f361809da20 2024-09-22T04:05:00.866 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.864+0000 7f4d117fa640 1 -- 172.21.15.3:0/1687569659 <== mon.0 v2:172.21.15.3:3300/0 6 ==== osd_map(716..716 src has 590..716) ==== 628+0+0 (crc 0 0 0) 0x7f4cfc0077b0 con 0x16bd9c0 2024-09-22T04:05:00.866 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.864+0000 7f3606ffd640 1 -- 172.21.15.3:0/1407891186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=717}) -- 0x7f35f8045d00 con 0x7f361809da20 2024-09-22T04:05:00.866 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.864+0000 7f4d117fa640 1 -- 172.21.15.3:0/1687569659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=717}) -- 0x7f4cf4043d90 con 0x16bd9c0 2024-09-22T04:05:00.993 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:00.992+0000 7f361cb51640 1 -- 172.21.15.3:0/1407891186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_bytes", "val": "0"} v 0) -- 0x7f36180697f0 con 0x7f361809da20 2024-09-22T04:05:01.874 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:01.872+0000 7f3606ffd640 1 -- 172.21.15.3:0/1407891186 <== mon.0 v2:172.21.15.3:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 v717) ==== 217+0+0 (crc 0 0 0) 0x7f360800e760 con 0x7f361809da20 2024-09-22T04:05:01.874 INFO:tasks.workunit.client.0.smithi003.stderr:set-quota max_bytes = 0 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 2024-09-22T04:05:01.874 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:01.873+0000 7f3606ffd640 1 -- 172.21.15.3:0/1407891186 <== mon.0 v2:172.21.15.3:3300/0 10 ==== osd_map(717..717 src has 590..717) ==== 628+0+0 (crc 0 0 0) 0x7f3608001fd0 con 0x7f361809da20 2024-09-22T04:05:01.874 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:01.873+0000 7f4d117fa640 1 -- 172.21.15.3:0/1687569659 <== mon.0 v2:172.21.15.3:3300/0 7 ==== osd_map(717..717 src has 590..717) ==== 628+0+0 (crc 0 0 0) 0x7f4cfc009250 con 0x16bd9c0 2024-09-22T04:05:01.874 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:01.873+0000 7f3606ffd640 1 -- 172.21.15.3:0/1407891186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=718}) -- 0x7f35f8045e20 con 0x7f361809da20 2024-09-22T04:05:01.875 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:01.873+0000 7f4d117fa640 1 -- 172.21.15.3:0/1687569659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=718}) -- 0x7f4cf4043fb0 con 0x16bd9c0 2024-09-22T04:05:01.877 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:01.875+0000 7f361cb51640 1 -- 172.21.15.3:0/1407891186 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f35f803ead0 msgr2=0x7f35f8040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:01.877 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:01.875+0000 7f361cb51640 1 --2- 172.21.15.3:0/1407891186 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f35f803ead0 0x7f35f8040fe0 crc :-1 s=READY pgs=3848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:01.877 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:01.876+0000 7f361cb51640 1 -- 172.21.15.3:0/1407891186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f361809da20 msgr2=0x7f36181208a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:01.877 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:01.876+0000 7f361cb51640 1 --2- 172.21.15.3:0/1407891186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f361809da20 0x7f36181208a0 crc :-1 s=READY pgs=12929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:01.877 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:01.876+0000 7f361cb51640 1 -- 172.21.15.3:0/1407891186 shutdown_connections 2024-09-22T04:05:01.877 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:01.876+0000 7f361cb51640 1 --2- 172.21.15.3:0/1407891186 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f35f803ead0 0x7f35f8040fe0 unknown :-1 s=CLOSED pgs=3848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:01.877 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:01.876+0000 7f361cb51640 1 --2- 172.21.15.3:0/1407891186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f361809da20 0x7f36181208a0 unknown :-1 s=CLOSED pgs=12929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:01.877 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:01.876+0000 7f361cb51640 1 -- 172.21.15.3:0/1407891186 >> 172.21.15.3:0/1407891186 conn(0x7f36180a6d10 msgr2=0x7f36180a4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:01.877 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:01.876+0000 7f361cb51640 1 -- 172.21.15.3:0/1407891186 shutdown_connections 2024-09-22T04:05:01.877 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:01.876+0000 7f361cb51640 1 -- 172.21.15.3:0/1407891186 wait complete. 2024-09-22T04:05:01.893 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool set-quota 7c1c124a-aa8e-411d-9731-1f1389755458 max_objects 0 2024-09-22T04:05:02.002 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:01.992+0000 7f2a1e46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:02.003 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.002+0000 7f2a1e46c640 1 Processor -- start 2024-09-22T04:05:02.003 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.002+0000 7f2a1e46c640 1 -- start start 2024-09-22T04:05:02.006 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.004+0000 7f2a1e46c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2a18135cb0 0x7f2a181380d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:02.006 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.004+0000 7f2a1e46c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a18132940 con 0x7f2a18132ba0 2024-09-22T04:05:02.006 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.004+0000 7f2a1e46c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2a1806f7a0 con 0x7f2a18135cb0 2024-09-22T04:05:02.006 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.004+0000 7f2a177fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2a18135cb0 0x7f2a181380d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:02.006 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.004+0000 7f2a17fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2a18132ba0 0x7f2a18132fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51614/0 (socket says 172.21.15.3:51614) 2024-09-22T04:05:02.006 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.004+0000 7f2a17fff640 1 -- 172.21.15.3:0/652962790 learned_addr learned my addr 172.21.15.3:0/652962790 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T04:05:02.006 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.004+0000 7f2a177fe640 1 -- 172.21.15.3:0/652962790 >> v1:172.21.15.3:6789/0 conn(0x7f2a18132ba0 legacy=0x7f2a18132fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:02.006 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.004+0000 7f2a177fe640 1 -- 172.21.15.3:0/652962790 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1806ebc0 con 0x7f2a18135cb0 2024-09-22T04:05:02.006 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.004+0000 7f2a16ffd640 1 -- 172.21.15.3:0/652962790 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 758892104 0 0) 0x7f2a18132940 con 0x7f2a18132ba0 2024-09-22T04:05:02.006 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.004+0000 7f2a177fe640 1 --2- 172.21.15.3:0/652962790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18135cb0 0x7f2a181380d0 crc :-1 s=READY pgs=12931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3543d5811be0881 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:02.007 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.004+0000 7f2a16ffd640 1 -- 172.21.15.3:0/652962790 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2a0c010570 con 0x7f2a18135cb0 2024-09-22T04:05:02.007 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.005+0000 7f2a16ffd640 1 -- 172.21.15.3:0/652962790 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2a0c0106d0 con 0x7f2a18135cb0 2024-09-22T04:05:02.007 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.005+0000 7f2a16ffd640 1 -- 172.21.15.3:0/652962790 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2a0c010970 con 0x7f2a18135cb0 2024-09-22T04:05:02.008 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.005+0000 7f2a1e46c640 1 -- 172.21.15.3:0/652962790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18135cb0 msgr2=0x7f2a181380d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:02.008 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.005+0000 7f2a1e46c640 1 --2- 172.21.15.3:0/652962790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18135cb0 0x7f2a181380d0 crc :-1 s=READY pgs=12931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:02.008 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.005+0000 7f2a1e46c640 1 -- 172.21.15.3:0/652962790 shutdown_connections 2024-09-22T04:05:02.008 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.005+0000 7f2a1e46c640 1 --2- 172.21.15.3:0/652962790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18135cb0 0x7f2a181380d0 unknown :-1 s=CLOSED pgs=12931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:02.008 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.005+0000 7f2a1e46c640 1 -- 172.21.15.3:0/652962790 >> 172.21.15.3:0/652962790 conn(0x7f2a180a0f60 msgr2=0x7f2a180a1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:02.008 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.005+0000 7f2a1e46c640 1 -- 172.21.15.3:0/652962790 shutdown_connections 2024-09-22T04:05:02.008 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.005+0000 7f2a1e46c640 1 -- 172.21.15.3:0/652962790 wait complete. 2024-09-22T04:05:02.008 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.006+0000 7f2a1e46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:02.009 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.007+0000 7f2a1e46c640 1 Processor -- start 2024-09-22T04:05:02.009 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.007+0000 7f2a1e46c640 1 -- start start 2024-09-22T04:05:02.011 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.009+0000 7f2a1e46c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18132ba0 0x7f2a181294d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:02.011 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.009+0000 7f2a1e46c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1812b030 con 0x7f2a18132ba0 2024-09-22T04:05:02.011 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.009+0000 7f2a17fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18132ba0 0x7f2a181294d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:02.011 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.009+0000 7f2a17fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18132ba0 0x7f2a181294d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36116/0 (socket says 172.21.15.3:36116) 2024-09-22T04:05:02.011 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.009+0000 7f2a17fff640 1 -- 172.21.15.3:0/3834731252 learned_addr learned my addr 172.21.15.3:0/3834731252 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:02.011 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.009+0000 7f2a17fff640 1 -- 172.21.15.3:0/3834731252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a18131190 con 0x7f2a18132ba0 2024-09-22T04:05:02.011 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.009+0000 7f2a17fff640 1 --2- 172.21.15.3:0/3834731252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18132ba0 0x7f2a181294d0 crc :-1 s=READY pgs=12932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:02.011 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.010+0000 7f2a14ff9640 1 -- 172.21.15.3:0/3834731252 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2a08004b80 con 0x7f2a18132ba0 2024-09-22T04:05:02.011 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.010+0000 7f2a14ff9640 1 -- 172.21.15.3:0/3834731252 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2a08004ce0 con 0x7f2a18132ba0 2024-09-22T04:05:02.012 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.010+0000 7f2a1e46c640 1 -- 172.21.15.3:0/3834731252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1812cb70 con 0x7f2a18132ba0 2024-09-22T04:05:02.012 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.010+0000 7f2a14ff9640 1 -- 172.21.15.3:0/3834731252 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2a08007120 con 0x7f2a18132ba0 2024-09-22T04:05:02.012 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.010+0000 7f2a1e46c640 1 -- 172.21.15.3:0/3834731252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a1812bff0 con 0x7f2a18132ba0 2024-09-22T04:05:02.013 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.011+0000 7f2a1e46c640 1 -- 172.21.15.3:0/3834731252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a1812b4a0 con 0x7f2a18132ba0 2024-09-22T04:05:02.013 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.011+0000 7f2a14ff9640 1 -- 172.21.15.3:0/3834731252 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f2a080072d0 con 0x7f2a18132ba0 2024-09-22T04:05:02.013 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.011+0000 7f2a14ff9640 1 --2- 172.21.15.3:0/3834731252 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f2a0403eb00 0x7f2a04041010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:02.013 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.011+0000 7f2a14ff9640 1 -- 172.21.15.3:0/3834731252 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(717..717 src has 590..717) ==== 3819+0+0 (crc 0 0 0) 0x7f2a0804fe40 con 0x7f2a18132ba0 2024-09-22T04:05:02.013 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.011+0000 7f2a14ff9640 1 -- 172.21.15.3:0/3834731252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=718}) -- 0x7f2a04044df0 con 0x7f2a18132ba0 2024-09-22T04:05:02.013 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.011+0000 7f2a177fe640 1 --2- 172.21.15.3:0/3834731252 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f2a0403eb00 0x7f2a04041010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:02.013 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.012+0000 7f2a177fe640 1 --2- 172.21.15.3:0/3834731252 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f2a0403eb00 0x7f2a04041010 crc :-1 s=READY pgs=3849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:02.017 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.016+0000 7f2a14ff9640 1 -- 172.21.15.3:0/3834731252 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f2a0800f420 con 0x7f2a18132ba0 2024-09-22T04:05:02.205 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.204+0000 7f2a1e46c640 1 -- 172.21.15.3:0/3834731252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_objects", "val": "0"} v 0) -- 0x7f2a1809fea0 con 0x7f2a18132ba0 2024-09-22T04:05:02.875 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.874+0000 7f2a14ff9640 1 -- 172.21.15.3:0/3834731252 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 v718) ==== 221+0+0 (crc 0 0 0) 0x7f2a08006740 con 0x7f2a18132ba0 2024-09-22T04:05:02.876 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.874+0000 7f4d117fa640 1 -- 172.21.15.3:0/1687569659 <== mon.0 v2:172.21.15.3:3300/0 8 ==== osd_map(718..718 src has 590..718) ==== 628+0+0 (crc 0 0 0) 0x7f4cfc009430 con 0x16bd9c0 2024-09-22T04:05:02.876 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.874+0000 7f4d117fa640 1 -- 172.21.15.3:0/1687569659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=719}) -- 0x7f4cf40441a0 con 0x16bd9c0 2024-09-22T04:05:02.876 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.874+0000 7f2a14ff9640 1 -- 172.21.15.3:0/3834731252 <== mon.0 v2:172.21.15.3:3300/0 8 ==== osd_map(718..718 src has 590..718) ==== 628+0+0 (crc 0 0 0) 0x7f2a0800cdc0 con 0x7f2a18132ba0 2024-09-22T04:05:02.876 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:02.874+0000 7f2a14ff9640 1 -- 172.21.15.3:0/3834731252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=719}) -- 0x7f2a04045d70 con 0x7f2a18132ba0 2024-09-22T04:05:03.002 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.001+0000 7f2a1e46c640 1 -- 172.21.15.3:0/3834731252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_objects", "val": "0"} v 0) -- 0x7f2a180008d0 con 0x7f2a18132ba0 2024-09-22T04:05:03.395 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.393+0000 7f2a14ff9640 1 -- 172.21.15.3:0/3834731252 <== mon.0 v2:172.21.15.3:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 v719) ==== 221+0+0 (crc 0 0 0) 0x7f2a0800a0c0 con 0x7f2a18132ba0 2024-09-22T04:05:03.395 INFO:tasks.workunit.client.0.smithi003.stderr:set-quota max_objects = 0 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 2024-09-22T04:05:03.395 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.393+0000 7f2a14ff9640 1 -- 172.21.15.3:0/3834731252 <== mon.0 v2:172.21.15.3:3300/0 10 ==== osd_map(719..719 src has 590..719) ==== 628+0+0 (crc 0 0 0) 0x7f2a0800dc70 con 0x7f2a18132ba0 2024-09-22T04:05:03.395 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.393+0000 7f4d117fa640 1 -- 172.21.15.3:0/1687569659 <== mon.0 v2:172.21.15.3:3300/0 9 ==== osd_map(719..719 src has 590..719) ==== 628+0+0 (crc 0 0 0) 0x7f4cfc00cba0 con 0x16bd9c0 2024-09-22T04:05:03.395 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.393+0000 7f4d117fa640 1 -- 172.21.15.3:0/1687569659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=720}) -- 0x7f4cf40441a0 con 0x16bd9c0 2024-09-22T04:05:03.395 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.393+0000 7f2a14ff9640 1 -- 172.21.15.3:0/3834731252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=720}) -- 0x7f2a04045f00 con 0x7f2a18132ba0 2024-09-22T04:05:03.395 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.393+0000 7f4d117fa640 1 -- 172.21.15.3:0/1687569659 --> v2:172.21.15.123:6806/4206675901 -- osd_op(unknown.0.0:1 199.6 199:6b5d76a5:::two:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e719) -- 0x7f4cf40443f0 con 0x16be4e0 2024-09-22T04:05:03.397 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.396+0000 7f2a1e46c640 1 -- 172.21.15.3:0/3834731252 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f2a0403eb00 msgr2=0x7f2a04041010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:03.397 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.396+0000 7f2a1e46c640 1 --2- 172.21.15.3:0/3834731252 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f2a0403eb00 0x7f2a04041010 crc :-1 s=READY pgs=3849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.398 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.396+0000 7f2a1e46c640 1 -- 172.21.15.3:0/3834731252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18132ba0 msgr2=0x7f2a181294d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:03.398 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.396+0000 7f2a1e46c640 1 --2- 172.21.15.3:0/3834731252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18132ba0 0x7f2a181294d0 crc :-1 s=READY pgs=12932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.398 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.396+0000 7f2a1e46c640 1 -- 172.21.15.3:0/3834731252 shutdown_connections 2024-09-22T04:05:03.398 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.396+0000 7f2a1e46c640 1 --2- 172.21.15.3:0/3834731252 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f2a0403eb00 0x7f2a04041010 unknown :-1 s=CLOSED pgs=3849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.398 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.396+0000 7f2a1e46c640 1 --2- 172.21.15.3:0/3834731252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a18132ba0 0x7f2a181294d0 unknown :-1 s=CLOSED pgs=12932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.398 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.396+0000 7f2a1e46c640 1 -- 172.21.15.3:0/3834731252 >> 172.21.15.3:0/3834731252 conn(0x7f2a180a0f60 msgr2=0x7f2a180a1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:03.398 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.396+0000 7f2a1e46c640 1 -- 172.21.15.3:0/3834731252 shutdown_connections 2024-09-22T04:05:03.398 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.396+0000 7f2a1e46c640 1 -- 172.21.15.3:0/3834731252 wait complete. 2024-09-22T04:05:03.412 INFO:tasks.workunit.client.0.smithi003.stderr:+ wait 95989 2024-09-22T04:05:03.470 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.468+0000 7f4d1959c640 1 -- 172.21.15.3:0/1687569659 <== osd.3 v2:172.21.15.123:6806/4206675901 1 ==== osd_op_reply(1 two [writefull 0~12875] v719'2 uv2 ondisk = 0) ==== 147+0+0 (crc 0 0 0) 0x7f4d0c00a430 con 0x16be4e0 2024-09-22T04:05:03.470 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.468+0000 7f4d20c76ec0 1 -- 172.21.15.3:0/1687569659 >> v2:172.21.15.123:6806/4206675901 conn(0x16be4e0 msgr2=0x1be9840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:03.470 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.468+0000 7f4d20c76ec0 1 --2- 172.21.15.3:0/1687569659 >> v2:172.21.15.123:6806/4206675901 conn(0x16be4e0 0x1be9840 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.470 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.468+0000 7f4d20c76ec0 1 -- 172.21.15.3:0/1687569659 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4cf403d360 msgr2=0x7f4cf403f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:03.470 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.468+0000 7f4d20c76ec0 1 --2- 172.21.15.3:0/1687569659 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4cf403d360 0x7f4cf403f870 crc :-1 s=READY pgs=3847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.470 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.468+0000 7f4d20c76ec0 1 -- 172.21.15.3:0/1687569659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16bd9c0 msgr2=0x173b9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:03.470 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.468+0000 7f4d20c76ec0 1 --2- 172.21.15.3:0/1687569659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16bd9c0 0x173b9c0 crc :-1 s=READY pgs=12926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.470 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.468+0000 7f4d20c76ec0 1 -- 172.21.15.3:0/1687569659 shutdown_connections 2024-09-22T04:05:03.471 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.468+0000 7f4d20c76ec0 1 --2- 172.21.15.3:0/1687569659 >> v2:172.21.15.123:6806/4206675901 conn(0x16be4e0 0x1be9840 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.471 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.468+0000 7f4d20c76ec0 1 --2- 172.21.15.3:0/1687569659 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4cf403d360 0x7f4cf403f870 unknown :-1 s=CLOSED pgs=3847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.471 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.468+0000 7f4d20c76ec0 1 --2- 172.21.15.3:0/1687569659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x16bd9c0 0x173b9c0 unknown :-1 s=CLOSED pgs=12926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.471 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.468+0000 7f4d20c76ec0 1 -- 172.21.15.3:0/1687569659 >> 172.21.15.3:0/1687569659 conn(0x15e2770 msgr2=0x1751160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:03.471 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.468+0000 7f4d20c76ec0 1 -- 172.21.15.3:0/1687569659 shutdown_connections 2024-09-22T04:05:03.471 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.468+0000 7f4d20c76ec0 1 -- 172.21.15.3:0/1687569659 wait complete. 2024-09-22T04:05:03.472 INFO:tasks.workunit.client.0.smithi003.stderr:+ '[' 0 -ne 0 ']' 2024-09-22T04:05:03.472 INFO:tasks.workunit.client.0.smithi003.stderr:+ true 2024-09-22T04:05:03.472 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p 7c1c124a-aa8e-411d-9731-1f1389755458 put three /etc/passwd 2024-09-22T04:05:03.510 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.508+0000 7f3153a91ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:03.516 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.514+0000 7f3153a91ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:03.516 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.514+0000 7f3153a91ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:03.516 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.515+0000 7f3153a91ec0 1 Processor -- start 2024-09-22T04:05:03.516 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.515+0000 7f3153a91ec0 1 -- start start 2024-09-22T04:05:03.519 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.517+0000 7f3153a91ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x67af80 0x79c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:03.519 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.517+0000 7f3153a91ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5e6520 con 0x67af80 2024-09-22T04:05:03.519 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.517+0000 7f31477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x67af80 0x79c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:03.519 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.517+0000 7f31477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x67af80 0x79c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36140/0 (socket says 172.21.15.3:36140) 2024-09-22T04:05:03.519 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.517+0000 7f31477fe640 1 -- 172.21.15.3:0/798378699 learned_addr learned my addr 172.21.15.3:0/798378699 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:03.519 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.517+0000 7f31477fe640 1 -- 172.21.15.3:0/798378699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x582e10 con 0x67af80 2024-09-22T04:05:03.519 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.517+0000 7f31477fe640 1 --2- 172.21.15.3:0/798378699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x67af80 0x79c8a0 crc :-1 s=READY pgs=12935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8c1f1c88704a27 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:03.519 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.518+0000 7f3145ffb640 1 -- 172.21.15.3:0/798378699 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3134034040 con 0x67af80 2024-09-22T04:05:03.519 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.518+0000 7f3145ffb640 1 -- 172.21.15.3:0/798378699 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f313402f0b0 con 0x67af80 2024-09-22T04:05:03.519 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.518+0000 7f3145ffb640 1 -- 172.21.15.3:0/798378699 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f313402f350 con 0x67af80 2024-09-22T04:05:03.520 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.518+0000 7f3153a91ec0 1 -- 172.21.15.3:0/798378699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x67af80 msgr2=0x79c8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:03.520 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.518+0000 7f3153a91ec0 1 --2- 172.21.15.3:0/798378699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x67af80 0x79c8a0 crc :-1 s=READY pgs=12935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.520 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.518+0000 7f3153a91ec0 1 -- 172.21.15.3:0/798378699 shutdown_connections 2024-09-22T04:05:03.521 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.518+0000 7f3153a91ec0 1 --2- 172.21.15.3:0/798378699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x67af80 0x79c8a0 unknown :-1 s=CLOSED pgs=12935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.521 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.518+0000 7f3153a91ec0 1 -- 172.21.15.3:0/798378699 >> 172.21.15.3:0/798378699 conn(0x5a6770 msgr2=0x67d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:03.521 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.518+0000 7f3153a91ec0 1 -- 172.21.15.3:0/798378699 shutdown_connections 2024-09-22T04:05:03.521 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.518+0000 7f3153a91ec0 1 -- 172.21.15.3:0/798378699 wait complete. 2024-09-22T04:05:03.521 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.519+0000 7f3153a91ec0 1 Processor -- start 2024-09-22T04:05:03.521 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.519+0000 7f3153a91ec0 1 -- start start 2024-09-22T04:05:03.522 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.521+0000 7f3153a91ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x67af80 0x700fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:03.523 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.521+0000 7f3153a91ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5e4280 con 0x67af80 2024-09-22T04:05:03.523 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.521+0000 7f31477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x67af80 0x700fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:03.523 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.521+0000 7f31477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x67af80 0x700fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36142/0 (socket says 172.21.15.3:36142) 2024-09-22T04:05:03.523 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.521+0000 7f31477fe640 1 -- 172.21.15.3:0/1771095244 learned_addr learned my addr 172.21.15.3:0/1771095244 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:03.523 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.521+0000 7f31477fe640 1 -- 172.21.15.3:0/1771095244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x574920 con 0x67af80 2024-09-22T04:05:03.523 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.521+0000 7f31477fe640 1 --2- 172.21.15.3:0/1771095244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x67af80 0x700fd0 crc :-1 s=READY pgs=12936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:03.523 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.521+0000 7f3144ff9640 1 -- 172.21.15.3:0/1771095244 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3134034040 con 0x67af80 2024-09-22T04:05:03.523 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.521+0000 7f3144ff9640 1 -- 172.21.15.3:0/1771095244 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3134035040 con 0x67af80 2024-09-22T04:05:03.523 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.521+0000 7f3144ff9640 1 -- 172.21.15.3:0/1771095244 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f313402f350 con 0x67af80 2024-09-22T04:05:03.523 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.521+0000 7f3153a91ec0 1 -- 172.21.15.3:0/1771095244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x574380 con 0x67af80 2024-09-22T04:05:03.523 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.522+0000 7f3153a91ec0 1 -- 172.21.15.3:0/1771095244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x79dfc0 con 0x67af80 2024-09-22T04:05:03.524 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.522+0000 7f3144ff9640 1 -- 172.21.15.3:0/1771095244 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f3134003040 con 0x67af80 2024-09-22T04:05:03.524 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.523+0000 7f3144ff9640 1 --2- 172.21.15.3:0/1771095244 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f312403d360 0x7f312403f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:03.524 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.523+0000 7f3144ff9640 1 -- 172.21.15.3:0/1771095244 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(719..719 src has 590..719) ==== 3819+0+0 (crc 0 0 0) 0x7f31340737a0 con 0x67af80 2024-09-22T04:05:03.524 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.523+0000 7f3153a91ec0 1 --2- 172.21.15.3:0/1771095244 >> v2:172.21.15.123:6806/4206675901 conn(0x6830f0 0x714470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:03.524 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.523+0000 7f3153a91ec0 1 -- 172.21.15.3:0/1771095244 --> v2:172.21.15.123:6806/4206675901 -- osd_op(unknown.0.0:1 199.3 199:c57798d7:::three:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e719) -- 0x704970 con 0x6830f0 2024-09-22T04:05:03.524 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.523+0000 7f3146ffd640 1 --2- 172.21.15.3:0/1771095244 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f312403d360 0x7f312403f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:03.525 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.524+0000 7f3146ffd640 1 --2- 172.21.15.3:0/1771095244 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f312403d360 0x7f312403f870 crc :-1 s=READY pgs=3850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:03.525 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.524+0000 7f3147fff640 1 --2- 172.21.15.3:0/1771095244 >> v2:172.21.15.123:6806/4206675901 conn(0x6830f0 0x714470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:05:03.527 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.526+0000 7f3147fff640 1 --2- 172.21.15.3:0/1771095244 >> v2:172.21.15.123:6806/4206675901 conn(0x6830f0 0x714470 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:03.563 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.562+0000 7f3147fff640 1 -- 172.21.15.3:0/1771095244 <== osd.3 v2:172.21.15.123:6806/4206675901 1 ==== osd_op_reply(1 three [writefull 0~12875] v719'2 uv2 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f313c00c430 con 0x6830f0 2024-09-22T04:05:03.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.562+0000 7f3153a91ec0 1 -- 172.21.15.3:0/1771095244 >> v2:172.21.15.123:6806/4206675901 conn(0x6830f0 msgr2=0x714470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:03.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.562+0000 7f3153a91ec0 1 --2- 172.21.15.3:0/1771095244 >> v2:172.21.15.123:6806/4206675901 conn(0x6830f0 0x714470 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.562+0000 7f3153a91ec0 1 -- 172.21.15.3:0/1771095244 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f312403d360 msgr2=0x7f312403f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:03.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.562+0000 7f3153a91ec0 1 --2- 172.21.15.3:0/1771095244 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f312403d360 0x7f312403f870 crc :-1 s=READY pgs=3850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.562+0000 7f3153a91ec0 1 -- 172.21.15.3:0/1771095244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x67af80 msgr2=0x700fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:03.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.562+0000 7f3153a91ec0 1 --2- 172.21.15.3:0/1771095244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x67af80 0x700fd0 crc :-1 s=READY pgs=12936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.562+0000 7f3153a91ec0 1 -- 172.21.15.3:0/1771095244 shutdown_connections 2024-09-22T04:05:03.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.562+0000 7f3153a91ec0 1 --2- 172.21.15.3:0/1771095244 >> v2:172.21.15.123:6806/4206675901 conn(0x6830f0 0x714470 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.562+0000 7f3153a91ec0 1 --2- 172.21.15.3:0/1771095244 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f312403d360 0x7f312403f870 unknown :-1 s=CLOSED pgs=3850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.562+0000 7f3153a91ec0 1 --2- 172.21.15.3:0/1771095244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x67af80 0x700fd0 unknown :-1 s=CLOSED pgs=12936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.562+0000 7f3153a91ec0 1 -- 172.21.15.3:0/1771095244 >> 172.21.15.3:0/1771095244 conn(0x5a6770 msgr2=0x79c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:03.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.562+0000 7f3153a91ec0 1 -- 172.21.15.3:0/1771095244 shutdown_connections 2024-09-22T04:05:03.564 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.562+0000 7f3153a91ec0 1 -- 172.21.15.3:0/1771095244 wait complete. 2024-09-22T04:05:03.567 INFO:tasks.workunit.client.0.smithi003.stderr:++ uuidgen 2024-09-22T04:05:03.568 INFO:tasks.workunit.client.0.smithi003.stderr:+ pp=dd65dfbd-105b-4a15-8dc2-717b17ae7efb 2024-09-22T04:05:03.568 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool create dd65dfbd-105b-4a15-8dc2-717b17ae7efb 12 2024-09-22T04:05:03.669 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.660+0000 7fcd4dd11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:03.670 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.669+0000 7fcd4dd11640 1 Processor -- start 2024-09-22T04:05:03.670 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.669+0000 7fcd4dd11640 1 -- start start 2024-09-22T04:05:03.672 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.670+0000 7fcd4dd11640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcd481335b0 0x7fcd48137c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:03.672 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.670+0000 7fcd4dd11640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd4806f020 con 0x7fcd48132a90 2024-09-22T04:05:03.672 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.670+0000 7fcd4dd11640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcd4806f720 con 0x7fcd481335b0 2024-09-22T04:05:03.673 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.670+0000 7fcd477fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcd48132a90 0x7fcd48132ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51622/0 (socket says 172.21.15.3:51622) 2024-09-22T04:05:03.673 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.670+0000 7fcd477fe640 1 -- 172.21.15.3:0/3110289543 learned_addr learned my addr 172.21.15.3:0/3110289543 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T04:05:03.673 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.670+0000 7fcd46ffd640 1 --2- 172.21.15.3:0/3110289543 >> v2:172.21.15.3:3300/0 conn(0x7fcd481335b0 0x7fcd48137c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:03.673 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.671+0000 7fcd467fc640 1 -- 172.21.15.3:0/3110289543 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2557124040 0 0) 0x7fcd4806f020 con 0x7fcd48132a90 2024-09-22T04:05:03.673 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.671+0000 7fcd467fc640 1 -- 172.21.15.3:0/3110289543 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd30005790 con 0x7fcd48132a90 2024-09-22T04:05:03.673 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.671+0000 7fcd46ffd640 1 -- 172.21.15.3:0/3110289543 >> v1:172.21.15.3:6789/0 conn(0x7fcd48132a90 legacy=0x7fcd48132ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:03.673 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.671+0000 7fcd46ffd640 1 -- 172.21.15.3:0/3110289543 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4806e610 con 0x7fcd481335b0 2024-09-22T04:05:03.673 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.671+0000 7fcd46ffd640 1 --2- 172.21.15.3:0/3110289543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd481335b0 0x7fcd48137c90 crc :-1 s=READY pgs=12938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d78f7614b8fbcc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:03.673 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.671+0000 7fcd467fc640 1 -- 172.21.15.3:0/3110289543 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcd3c009d90 con 0x7fcd481335b0 2024-09-22T04:05:03.673 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.671+0000 7fcd467fc640 1 -- 172.21.15.3:0/3110289543 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fcd3c00e230 con 0x7fcd481335b0 2024-09-22T04:05:03.673 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.671+0000 7fcd467fc640 1 -- 172.21.15.3:0/3110289543 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcd3c00e4d0 con 0x7fcd481335b0 2024-09-22T04:05:03.674 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.672+0000 7fcd4dd11640 1 -- 172.21.15.3:0/3110289543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd481335b0 msgr2=0x7fcd48137c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:03.674 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.672+0000 7fcd4dd11640 1 --2- 172.21.15.3:0/3110289543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd481335b0 0x7fcd48137c90 crc :-1 s=READY pgs=12938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.675 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.672+0000 7fcd4dd11640 1 -- 172.21.15.3:0/3110289543 shutdown_connections 2024-09-22T04:05:03.675 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.672+0000 7fcd4dd11640 1 --2- 172.21.15.3:0/3110289543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd481335b0 0x7fcd48137c90 unknown :-1 s=CLOSED pgs=12938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:03.675 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.672+0000 7fcd4dd11640 1 -- 172.21.15.3:0/3110289543 >> 172.21.15.3:0/3110289543 conn(0x7fcd480a28c0 msgr2=0x7fcd480a2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:03.675 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.672+0000 7fcd4dd11640 1 -- 172.21.15.3:0/3110289543 shutdown_connections 2024-09-22T04:05:03.675 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.672+0000 7fcd4dd11640 1 -- 172.21.15.3:0/3110289543 wait complete. 2024-09-22T04:05:03.675 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.672+0000 7fcd4dd11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:03.675 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.674+0000 7fcd4dd11640 1 Processor -- start 2024-09-22T04:05:03.676 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.674+0000 7fcd4dd11640 1 -- start start 2024-09-22T04:05:03.677 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.676+0000 7fcd4dd11640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd48132a90 0x7fcd4811d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:03.677 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.676+0000 7fcd4dd11640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4811d360 con 0x7fcd48132a90 2024-09-22T04:05:03.677 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.676+0000 7fcd477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd48132a90 0x7fcd4811d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:03.677 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.676+0000 7fcd477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd48132a90 0x7fcd4811d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36158/0 (socket says 172.21.15.3:36158) 2024-09-22T04:05:03.677 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.676+0000 7fcd477fe640 1 -- 172.21.15.3:0/1556336646 learned_addr learned my addr 172.21.15.3:0/1556336646 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:03.677 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.676+0000 7fcd477fe640 1 -- 172.21.15.3:0/1556336646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4811e8d0 con 0x7fcd48132a90 2024-09-22T04:05:03.677 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.676+0000 7fcd477fe640 1 --2- 172.21.15.3:0/1556336646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd48132a90 0x7fcd4811d790 crc :-1 s=READY pgs=12939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:03.678 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.676+0000 7fcd2ffff640 1 -- 172.21.15.3:0/1556336646 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcd38005a60 con 0x7fcd48132a90 2024-09-22T04:05:03.679 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.676+0000 7fcd2ffff640 1 -- 172.21.15.3:0/1556336646 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fcd38005bc0 con 0x7fcd48132a90 2024-09-22T04:05:03.679 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.676+0000 7fcd4dd11640 1 -- 172.21.15.3:0/1556336646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd4811d050 con 0x7fcd48132a90 2024-09-22T04:05:03.679 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.676+0000 7fcd2ffff640 1 -- 172.21.15.3:0/1556336646 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcd38005e60 con 0x7fcd48132a90 2024-09-22T04:05:03.679 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.677+0000 7fcd4dd11640 1 -- 172.21.15.3:0/1556336646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd48120350 con 0x7fcd48132a90 2024-09-22T04:05:03.679 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.677+0000 7fcd4dd11640 1 -- 172.21.15.3:0/1556336646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd4811f280 con 0x7fcd48132a90 2024-09-22T04:05:03.679 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.678+0000 7fcd2ffff640 1 -- 172.21.15.3:0/1556336646 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fcd380070b0 con 0x7fcd48132a90 2024-09-22T04:05:03.680 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.678+0000 7fcd2ffff640 1 --2- 172.21.15.3:0/1556336646 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fcd3403ead0 0x7fcd34040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:03.680 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.678+0000 7fcd2ffff640 1 -- 172.21.15.3:0/1556336646 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(719..719 src has 590..719) ==== 3819+0+0 (crc 0 0 0) 0x7fcd3804fc90 con 0x7fcd48132a90 2024-09-22T04:05:03.680 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.678+0000 7fcd46ffd640 1 --2- 172.21.15.3:0/1556336646 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fcd3403ead0 0x7fcd34040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:03.680 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.679+0000 7fcd46ffd640 1 --2- 172.21.15.3:0/1556336646 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fcd3403ead0 0x7fcd34040fe0 crc :-1 s=READY pgs=3851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:03.684 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.683+0000 7fcd2ffff640 1 -- 172.21.15.3:0/1556336646 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7fcd3800dc60 con 0x7fcd48132a90 2024-09-22T04:05:03.871 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:03.870+0000 7fcd4dd11640 1 -- 172.21.15.3:0/1556336646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "pg_num": 12} v 0) -- 0x7fcd48120f10 con 0x7fcd48132a90 2024-09-22T04:05:04.404 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.403+0000 7fcd2ffff640 1 -- 172.21.15.3:0/1556336646 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "pg_num": 12}]=0 pool 'dd65dfbd-105b-4a15-8dc2-717b17ae7efb' created v720) ==== 176+0+0 (crc 0 0 0) 0x7fcd3800d660 con 0x7fcd48132a90 2024-09-22T04:05:04.540 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.539+0000 7fcd4dd11640 1 -- 172.21.15.3:0/1556336646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "pg_num": 12} v 0) -- 0x7fcd4806f3c0 con 0x7fcd48132a90 2024-09-22T04:05:04.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.540+0000 7fcd2ffff640 1 -- 172.21.15.3:0/1556336646 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "pg_num": 12}]=0 pool 'dd65dfbd-105b-4a15-8dc2-717b17ae7efb' already exists v720) ==== 183+0+0 (crc 0 0 0) 0x7fcd3800d840 con 0x7fcd48132a90 2024-09-22T04:05:04.541 INFO:tasks.workunit.client.0.smithi003.stderr:pool 'dd65dfbd-105b-4a15-8dc2-717b17ae7efb' already exists 2024-09-22T04:05:04.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.542+0000 7fcd4dd11640 1 -- 172.21.15.3:0/1556336646 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fcd3403ead0 msgr2=0x7fcd34040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:04.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.542+0000 7fcd4dd11640 1 --2- 172.21.15.3:0/1556336646 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fcd3403ead0 0x7fcd34040fe0 crc :-1 s=READY pgs=3851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:04.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.542+0000 7fcd4dd11640 1 -- 172.21.15.3:0/1556336646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd48132a90 msgr2=0x7fcd4811d790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:04.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.542+0000 7fcd4dd11640 1 --2- 172.21.15.3:0/1556336646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd48132a90 0x7fcd4811d790 crc :-1 s=READY pgs=12939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:04.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.542+0000 7fcd4dd11640 1 -- 172.21.15.3:0/1556336646 shutdown_connections 2024-09-22T04:05:04.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.542+0000 7fcd4dd11640 1 --2- 172.21.15.3:0/1556336646 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fcd3403ead0 0x7fcd34040fe0 unknown :-1 s=CLOSED pgs=3851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:04.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.542+0000 7fcd4dd11640 1 --2- 172.21.15.3:0/1556336646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd48132a90 0x7fcd4811d790 unknown :-1 s=CLOSED pgs=12939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:04.544 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.542+0000 7fcd4dd11640 1 -- 172.21.15.3:0/1556336646 >> 172.21.15.3:0/1556336646 conn(0x7fcd480a28c0 msgr2=0x7fcd48130900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:04.544 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.542+0000 7fcd4dd11640 1 -- 172.21.15.3:0/1556336646 shutdown_connections 2024-09-22T04:05:04.544 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.542+0000 7fcd4dd11640 1 -- 172.21.15.3:0/1556336646 wait complete. 2024-09-22T04:05:04.553 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool application enable dd65dfbd-105b-4a15-8dc2-717b17ae7efb rados 2024-09-22T04:05:04.649 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.639+0000 7f1b69d51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:04.650 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.649+0000 7f1b69d51640 1 Processor -- start 2024-09-22T04:05:04.651 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.649+0000 7f1b69d51640 1 -- start start 2024-09-22T04:05:04.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.651+0000 7f1b69d51640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b6406fda0 0x7f1b641341d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:04.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.651+0000 7f1b69d51640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b640701b0 con 0x7f1b64134710 2024-09-22T04:05:04.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.651+0000 7f1b69d51640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1b6409f4d0 con 0x7f1b6406fda0 2024-09-22T04:05:04.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.651+0000 7f1b637fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b6406fda0 0x7f1b641341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:04.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.651+0000 7f1b637fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1b6406fda0 0x7f1b641341d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36162/0 (socket says 172.21.15.3:36162) 2024-09-22T04:05:04.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.651+0000 7f1b637fe640 1 -- 172.21.15.3:0/1442511163 learned_addr learned my addr 172.21.15.3:0/1442511163 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:04.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.651+0000 7f1b62ffd640 1 --1- 172.21.15.3:0/1442511163 >> v1:172.21.15.3:6789/0 conn(0x7f1b64134710 0x7f1b6406e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51626/0 (socket says 172.21.15.3:51626) 2024-09-22T04:05:04.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.651+0000 7f1b637fe640 1 -- 172.21.15.3:0/1442511163 >> v1:172.21.15.3:6789/0 conn(0x7f1b64134710 legacy=0x7f1b6406e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:04.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.652+0000 7f1b637fe640 1 -- 172.21.15.3:0/1442511163 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6409f1c0 con 0x7f1b6406fda0 2024-09-22T04:05:04.654 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.652+0000 7f1b637fe640 1 --2- 172.21.15.3:0/1442511163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6406fda0 0x7f1b641341d0 crc :-1 s=READY pgs=12941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6227775291b2504 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:04.654 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.652+0000 7f1b627fc640 1 -- 172.21.15.3:0/1442511163 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1b54009e50 con 0x7f1b6406fda0 2024-09-22T04:05:04.654 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.652+0000 7f1b627fc640 1 -- 172.21.15.3:0/1442511163 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f1b5400e230 con 0x7f1b6406fda0 2024-09-22T04:05:04.654 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.652+0000 7f1b627fc640 1 -- 172.21.15.3:0/1442511163 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1b5400e4d0 con 0x7f1b6406fda0 2024-09-22T04:05:04.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.653+0000 7f1b69d51640 1 -- 172.21.15.3:0/1442511163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6406fda0 msgr2=0x7f1b641341d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:04.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.653+0000 7f1b69d51640 1 --2- 172.21.15.3:0/1442511163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6406fda0 0x7f1b641341d0 crc :-1 s=READY pgs=12941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:04.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.653+0000 7f1b69d51640 1 -- 172.21.15.3:0/1442511163 shutdown_connections 2024-09-22T04:05:04.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.653+0000 7f1b69d51640 1 --2- 172.21.15.3:0/1442511163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6406fda0 0x7f1b641341d0 unknown :-1 s=CLOSED pgs=12941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:04.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.653+0000 7f1b69d51640 1 -- 172.21.15.3:0/1442511163 >> 172.21.15.3:0/1442511163 conn(0x7f1b640a3110 msgr2=0x7f1b640a3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:04.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.653+0000 7f1b69d51640 1 -- 172.21.15.3:0/1442511163 shutdown_connections 2024-09-22T04:05:04.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.653+0000 7f1b69d51640 1 -- 172.21.15.3:0/1442511163 wait complete. 2024-09-22T04:05:04.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.653+0000 7f1b69d51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:04.656 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.655+0000 7f1b69d51640 1 Processor -- start 2024-09-22T04:05:04.656 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.655+0000 7f1b69d51640 1 -- start start 2024-09-22T04:05:04.658 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.656+0000 7f1b69d51640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6406fda0 0x7f1b64131380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:04.658 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.656+0000 7f1b69d51640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1b64133690 con 0x7f1b6406fda0 2024-09-22T04:05:04.658 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.656+0000 7f1b637fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6406fda0 0x7f1b64131380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:04.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.656+0000 7f1b637fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6406fda0 0x7f1b64131380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36172/0 (socket says 172.21.15.3:36172) 2024-09-22T04:05:04.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.656+0000 7f1b637fe640 1 -- 172.21.15.3:0/1694330271 learned_addr learned my addr 172.21.15.3:0/1694330271 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:04.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.657+0000 7f1b637fe640 1 -- 172.21.15.3:0/1694330271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b64133e50 con 0x7f1b6406fda0 2024-09-22T04:05:04.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.657+0000 7f1b637fe640 1 --2- 172.21.15.3:0/1694330271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6406fda0 0x7f1b64131380 crc :-1 s=READY pgs=12942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:04.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.657+0000 7f1b4bfff640 1 -- 172.21.15.3:0/1694330271 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1b5400eda0 con 0x7f1b6406fda0 2024-09-22T04:05:04.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.657+0000 7f1b4bfff640 1 -- 172.21.15.3:0/1694330271 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f1b54015420 con 0x7f1b6406fda0 2024-09-22T04:05:04.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.657+0000 7f1b69d51640 1 -- 172.21.15.3:0/1694330271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6408e1c0 con 0x7f1b6406fda0 2024-09-22T04:05:04.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.657+0000 7f1b4bfff640 1 -- 172.21.15.3:0/1694330271 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1b540156c0 con 0x7f1b6406fda0 2024-09-22T04:05:04.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.657+0000 7f1b69d51640 1 -- 172.21.15.3:0/1694330271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b6409e810 con 0x7f1b6406fda0 2024-09-22T04:05:04.660 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.658+0000 7f1b69d51640 1 -- 172.21.15.3:0/1694330271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b6408eeb0 con 0x7f1b6406fda0 2024-09-22T04:05:04.660 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.658+0000 7f1b4bfff640 1 -- 172.21.15.3:0/1694330271 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f1b54015870 con 0x7f1b6406fda0 2024-09-22T04:05:04.660 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.658+0000 7f1b4bfff640 1 --2- 172.21.15.3:0/1694330271 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f1b5003ead0 0x7f1b50040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:04.660 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.659+0000 7f1b4bfff640 1 -- 172.21.15.3:0/1694330271 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(720..720 src has 590..720) ==== 4194+0+0 (crc 0 0 0) 0x7f1b54051430 con 0x7f1b6406fda0 2024-09-22T04:05:04.660 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.659+0000 7f1b62ffd640 1 --2- 172.21.15.3:0/1694330271 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f1b5003ead0 0x7f1b50040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:04.660 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.659+0000 7f1b62ffd640 1 --2- 172.21.15.3:0/1694330271 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f1b5003ead0 0x7f1b50040fe0 crc :-1 s=READY pgs=3852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:04.665 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.664+0000 7f1b4bfff640 1 -- 172.21.15.3:0/1694330271 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f1b54012c00 con 0x7f1b6406fda0 2024-09-22T04:05:04.859 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:04.857+0000 7f1b69d51640 1 -- 172.21.15.3:0/1694330271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "app": "rados"} v 0) -- 0x7f1b64134f10 con 0x7f1b6406fda0 2024-09-22T04:05:05.405 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:05.404+0000 7f1b4bfff640 1 -- 172.21.15.3:0/1694330271 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "app": "rados"}]=0 enabled application 'rados' on pool 'dd65dfbd-105b-4a15-8dc2-717b17ae7efb' v721) ==== 213+0+0 (crc 0 0 0) 0x7f1b54012330 con 0x7f1b6406fda0 2024-09-22T04:05:05.587 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:05.586+0000 7f1b69d51640 1 -- 172.21.15.3:0/1694330271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "app": "rados"} v 0) -- 0x7f1b6408e4b0 con 0x7f1b6406fda0 2024-09-22T04:05:06.409 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.408+0000 7f1b4bfff640 1 -- 172.21.15.3:0/1694330271 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "app": "rados"}]=0 enabled application 'rados' on pool 'dd65dfbd-105b-4a15-8dc2-717b17ae7efb' v722) ==== 213+0+0 (crc 0 0 0) 0x7f1b54012a20 con 0x7f1b6406fda0 2024-09-22T04:05:06.410 INFO:tasks.workunit.client.0.smithi003.stderr:enabled application 'rados' on pool 'dd65dfbd-105b-4a15-8dc2-717b17ae7efb' 2024-09-22T04:05:06.413 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.411+0000 7f1b69d51640 1 -- 172.21.15.3:0/1694330271 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f1b5003ead0 msgr2=0x7f1b50040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:06.413 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.411+0000 7f1b69d51640 1 --2- 172.21.15.3:0/1694330271 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f1b5003ead0 0x7f1b50040fe0 crc :-1 s=READY pgs=3852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:06.413 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.411+0000 7f1b69d51640 1 -- 172.21.15.3:0/1694330271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6406fda0 msgr2=0x7f1b64131380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:06.413 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.411+0000 7f1b69d51640 1 --2- 172.21.15.3:0/1694330271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6406fda0 0x7f1b64131380 crc :-1 s=READY pgs=12942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:06.413 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.411+0000 7f1b69d51640 1 -- 172.21.15.3:0/1694330271 shutdown_connections 2024-09-22T04:05:06.413 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.411+0000 7f1b69d51640 1 --2- 172.21.15.3:0/1694330271 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f1b5003ead0 0x7f1b50040fe0 unknown :-1 s=CLOSED pgs=3852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:06.413 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.411+0000 7f1b69d51640 1 --2- 172.21.15.3:0/1694330271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b6406fda0 0x7f1b64131380 unknown :-1 s=CLOSED pgs=12942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:06.413 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.411+0000 7f1b69d51640 1 -- 172.21.15.3:0/1694330271 >> 172.21.15.3:0/1694330271 conn(0x7f1b640a3110 msgr2=0x7f1b6412a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:06.413 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.412+0000 7f1b69d51640 1 -- 172.21.15.3:0/1694330271 shutdown_connections 2024-09-22T04:05:06.413 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.412+0000 7f1b69d51640 1 -- 172.21.15.3:0/1694330271 wait complete. 2024-09-22T04:05:06.428 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool set-quota dd65dfbd-105b-4a15-8dc2-717b17ae7efb max_objects 10 2024-09-22T04:05:06.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.532+0000 7fc897191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:06.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.542+0000 7fc897191640 1 Processor -- start 2024-09-22T04:05:06.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.542+0000 7fc897191640 1 -- start start 2024-09-22T04:05:06.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.543+0000 7fc897191640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc89006fda0 0x7fc8901341d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:06.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.543+0000 7fc897191640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8900701b0 con 0x7fc890134710 2024-09-22T04:05:06.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.543+0000 7fc897191640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc89009f4d0 con 0x7fc89006fda0 2024-09-22T04:05:06.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.543+0000 7fc894f2d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc89006fda0 0x7fc8901341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:06.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.543+0000 7fc88ffff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc890134710 0x7fc89006e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:59350/0 (socket says 172.21.15.3:59350) 2024-09-22T04:05:06.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.543+0000 7fc88ffff640 1 -- 172.21.15.3:0/2168595240 learned_addr learned my addr 172.21.15.3:0/2168595240 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T04:05:06.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.544+0000 7fc894f2d640 1 -- 172.21.15.3:0/2168595240 >> v1:172.21.15.3:6789/0 conn(0x7fc890134710 legacy=0x7fc89006e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:06.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.544+0000 7fc894f2d640 1 -- 172.21.15.3:0/2168595240 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89009f1c0 con 0x7fc89006fda0 2024-09-22T04:05:06.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.544+0000 7fc894f2d640 1 --2- 172.21.15.3:0/2168595240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc89006fda0 0x7fc8901341d0 crc :-1 s=READY pgs=12944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a252df1354b849 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:06.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.544+0000 7fc88f7fe640 1 -- 172.21.15.3:0/2168595240 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc878009d90 con 0x7fc89006fda0 2024-09-22T04:05:06.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.544+0000 7fc88f7fe640 1 -- 172.21.15.3:0/2168595240 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc87800e230 con 0x7fc89006fda0 2024-09-22T04:05:06.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.544+0000 7fc88f7fe640 1 -- 172.21.15.3:0/2168595240 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc87800e4d0 con 0x7fc89006fda0 2024-09-22T04:05:06.547 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.545+0000 7fc897191640 1 -- 172.21.15.3:0/2168595240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc89006fda0 msgr2=0x7fc8901341d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:06.547 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.545+0000 7fc897191640 1 --2- 172.21.15.3:0/2168595240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc89006fda0 0x7fc8901341d0 crc :-1 s=READY pgs=12944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:06.548 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.545+0000 7fc897191640 1 -- 172.21.15.3:0/2168595240 shutdown_connections 2024-09-22T04:05:06.548 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.545+0000 7fc897191640 1 --2- 172.21.15.3:0/2168595240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc89006fda0 0x7fc8901341d0 unknown :-1 s=CLOSED pgs=12944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:06.548 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.545+0000 7fc897191640 1 -- 172.21.15.3:0/2168595240 >> 172.21.15.3:0/2168595240 conn(0x7fc8900a3110 msgr2=0x7fc8900a3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:06.548 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.545+0000 7fc897191640 1 -- 172.21.15.3:0/2168595240 shutdown_connections 2024-09-22T04:05:06.548 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.545+0000 7fc897191640 1 -- 172.21.15.3:0/2168595240 wait complete. 2024-09-22T04:05:06.548 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.545+0000 7fc897191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:06.548 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.547+0000 7fc897191640 1 Processor -- start 2024-09-22T04:05:06.549 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.547+0000 7fc897191640 1 -- start start 2024-09-22T04:05:06.551 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.549+0000 7fc897191640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc89006fda0 0x7fc890131380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:06.551 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.549+0000 7fc897191640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc890133690 con 0x7fc89006fda0 2024-09-22T04:05:06.551 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.549+0000 7fc894f2d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc89006fda0 0x7fc890131380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:06.551 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.549+0000 7fc894f2d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc89006fda0 0x7fc890131380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50824/0 (socket says 172.21.15.3:50824) 2024-09-22T04:05:06.551 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.549+0000 7fc894f2d640 1 -- 172.21.15.3:0/1067693387 learned_addr learned my addr 172.21.15.3:0/1067693387 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:06.551 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.549+0000 7fc894f2d640 1 -- 172.21.15.3:0/1067693387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc890133e50 con 0x7fc89006fda0 2024-09-22T04:05:06.551 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.549+0000 7fc894f2d640 1 --2- 172.21.15.3:0/1067693387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc89006fda0 0x7fc890131380 crc :-1 s=READY pgs=12945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:06.551 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.549+0000 7fc88d7fa640 1 -- 172.21.15.3:0/1067693387 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc878014540 con 0x7fc89006fda0 2024-09-22T04:05:06.551 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.549+0000 7fc897191640 1 -- 172.21.15.3:0/1067693387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89008e1c0 con 0x7fc89006fda0 2024-09-22T04:05:06.551 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.549+0000 7fc88d7fa640 1 -- 172.21.15.3:0/1067693387 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc8780146a0 con 0x7fc89006fda0 2024-09-22T04:05:06.552 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.550+0000 7fc88d7fa640 1 -- 172.21.15.3:0/1067693387 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc878014940 con 0x7fc89006fda0 2024-09-22T04:05:06.552 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.550+0000 7fc897191640 1 -- 172.21.15.3:0/1067693387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89009e810 con 0x7fc89006fda0 2024-09-22T04:05:06.552 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.551+0000 7fc897191640 1 -- 172.21.15.3:0/1067693387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc89008eeb0 con 0x7fc89006fda0 2024-09-22T04:05:06.553 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.551+0000 7fc88d7fa640 1 -- 172.21.15.3:0/1067693387 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fc878014d50 con 0x7fc89006fda0 2024-09-22T04:05:06.553 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.551+0000 7fc88d7fa640 1 --2- 172.21.15.3:0/1067693387 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc87003ea80 0x7fc870040f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:06.553 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.551+0000 7fc88d7fa640 1 -- 172.21.15.3:0/1067693387 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(722..722 src has 590..722) ==== 4207+0+0 (crc 0 0 0) 0x7fc878019040 con 0x7fc89006fda0 2024-09-22T04:05:06.553 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.551+0000 7fc88ffff640 1 --2- 172.21.15.3:0/1067693387 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc87003ea80 0x7fc870040f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:06.553 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.552+0000 7fc88ffff640 1 --2- 172.21.15.3:0/1067693387 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc87003ea80 0x7fc870040f90 crc :-1 s=READY pgs=3853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:06.557 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.556+0000 7fc88d7fa640 1 -- 172.21.15.3:0/1067693387 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7fc878011380 con 0x7fc89006fda0 2024-09-22T04:05:06.744 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:06.743+0000 7fc897191640 1 -- 172.21.15.3:0/1067693387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "field": "max_objects", "val": "10"} v 0) -- 0x7fc890134f10 con 0x7fc89006fda0 2024-09-22T04:05:07.410 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:07.409+0000 7fc88d7fa640 1 -- 172.21.15.3:0/1067693387 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool dd65dfbd-105b-4a15-8dc2-717b17ae7efb v723) ==== 223+0+0 (crc 0 0 0) 0x7fc878018190 con 0x7fc89006fda0 2024-09-22T04:05:07.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:07.542+0000 7fc897191640 1 -- 172.21.15.3:0/1067693387 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "field": "max_objects", "val": "10"} v 0) -- 0x7fc89008e4b0 con 0x7fc89006fda0 2024-09-22T04:05:08.423 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:08.422+0000 7fc88d7fa640 1 -- 172.21.15.3:0/1067693387 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool dd65dfbd-105b-4a15-8dc2-717b17ae7efb v724) ==== 223+0+0 (crc 0 0 0) 0x7fc878018370 con 0x7fc89006fda0 2024-09-22T04:05:08.423 INFO:tasks.workunit.client.0.smithi003.stderr:set-quota max_objects = 10 for pool dd65dfbd-105b-4a15-8dc2-717b17ae7efb 2024-09-22T04:05:08.428 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:08.426+0000 7fc897191640 1 -- 172.21.15.3:0/1067693387 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc87003ea80 msgr2=0x7fc870040f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:08.428 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:08.426+0000 7fc897191640 1 --2- 172.21.15.3:0/1067693387 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc87003ea80 0x7fc870040f90 crc :-1 s=READY pgs=3853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:08.428 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:08.426+0000 7fc897191640 1 -- 172.21.15.3:0/1067693387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc89006fda0 msgr2=0x7fc890131380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:08.428 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:08.426+0000 7fc897191640 1 --2- 172.21.15.3:0/1067693387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc89006fda0 0x7fc890131380 crc :-1 s=READY pgs=12945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:08.428 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:08.427+0000 7fc897191640 1 -- 172.21.15.3:0/1067693387 shutdown_connections 2024-09-22T04:05:08.428 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:08.427+0000 7fc897191640 1 --2- 172.21.15.3:0/1067693387 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc87003ea80 0x7fc870040f90 unknown :-1 s=CLOSED pgs=3853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:08.428 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:08.427+0000 7fc897191640 1 --2- 172.21.15.3:0/1067693387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc89006fda0 0x7fc890131380 unknown :-1 s=CLOSED pgs=12945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:08.429 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:08.427+0000 7fc897191640 1 -- 172.21.15.3:0/1067693387 >> 172.21.15.3:0/1067693387 conn(0x7fc8900a3110 msgr2=0x7fc89012a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:08.429 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:08.427+0000 7fc897191640 1 -- 172.21.15.3:0/1067693387 shutdown_connections 2024-09-22T04:05:08.429 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:08.427+0000 7fc897191640 1 -- 172.21.15.3:0/1067693387 wait complete. 2024-09-22T04:05:08.442 INFO:tasks.workunit.client.0.smithi003.stderr:+ sleep 30 2024-09-22T04:05:38.444 INFO:tasks.workunit.client.0.smithi003.stderr:++ seq 1 10 2024-09-22T04:05:38.445 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:05:38.445 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p dd65dfbd-105b-4a15-8dc2-717b17ae7efb put obj1 /etc/passwd 2024-09-22T04:05:38.492 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.491+0000 7f8e07253ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.499 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.498+0000 7f8e07253ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.499 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.498+0000 7f8e07253ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.500 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.499+0000 7f8e07253ec0 1 Processor -- start 2024-09-22T04:05:38.500 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.499+0000 7f8e07253ec0 1 -- start start 2024-09-22T04:05:38.501 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.500+0000 7f8e07253ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1f5d8a0 0x1fe3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.501 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.500+0000 7f8e07253ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x1e30520 con 0x1f5d8a0 2024-09-22T04:05:38.501 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.500+0000 7f8dff39b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1f5d8a0 0x1fe3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.500+0000 7f8dff39b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1f5d8a0 0x1fe3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42476/0 (socket says 172.21.15.3:42476) 2024-09-22T04:05:38.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.500+0000 7f8dff39b640 1 -- 172.21.15.3:0/3434666223 learned_addr learned my addr 172.21.15.3:0/3434666223 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:38.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.501+0000 7f8dff39b640 1 -- 172.21.15.3:0/3434666223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1dcce10 con 0x1f5d8a0 2024-09-22T04:05:38.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.501+0000 7f8dff39b640 1 --2- 172.21.15.3:0/3434666223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1f5d8a0 0x1fe3600 crc :-1 s=READY pgs=12948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac96a2b0f9084cf server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.501+0000 7f8dfdb98640 1 -- 172.21.15.3:0/3434666223 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8de802db80 con 0x1f5d8a0 2024-09-22T04:05:38.502 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.501+0000 7f8dfdb98640 1 -- 172.21.15.3:0/3434666223 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8de802dce0 con 0x1f5d8a0 2024-09-22T04:05:38.503 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.501+0000 7f8dfdb98640 1 -- 172.21.15.3:0/3434666223 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8de8035420 con 0x1f5d8a0 2024-09-22T04:05:38.503 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.502+0000 7f8e07253ec0 1 -- 172.21.15.3:0/3434666223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1f5d8a0 msgr2=0x1fe3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.503 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.502+0000 7f8e07253ec0 1 --2- 172.21.15.3:0/3434666223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1f5d8a0 0x1fe3600 crc :-1 s=READY pgs=12948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.503 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.502+0000 7f8e07253ec0 1 -- 172.21.15.3:0/3434666223 shutdown_connections 2024-09-22T04:05:38.504 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.502+0000 7f8e07253ec0 1 --2- 172.21.15.3:0/3434666223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1f5d8a0 0x1fe3600 unknown :-1 s=CLOSED pgs=12948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.504 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.502+0000 7f8e07253ec0 1 -- 172.21.15.3:0/3434666223 >> 172.21.15.3:0/3434666223 conn(0x1df0770 msgr2=0x1ecf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:38.504 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.502+0000 7f8e07253ec0 1 -- 172.21.15.3:0/3434666223 shutdown_connections 2024-09-22T04:05:38.504 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.502+0000 7f8e07253ec0 1 -- 172.21.15.3:0/3434666223 wait complete. 2024-09-22T04:05:38.504 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.503+0000 7f8e07253ec0 1 Processor -- start 2024-09-22T04:05:38.504 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.503+0000 7f8e07253ec0 1 -- start start 2024-09-22T04:05:38.506 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.505+0000 7f8e07253ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1f5d8a0 0x1ff5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.506 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.505+0000 7f8e07253ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x1e2e280 con 0x1f5d8a0 2024-09-22T04:05:38.506 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.505+0000 7f8dff39b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1f5d8a0 0x1ff5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.506 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.505+0000 7f8dff39b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1f5d8a0 0x1ff5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42482/0 (socket says 172.21.15.3:42482) 2024-09-22T04:05:38.506 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.505+0000 7f8dff39b640 1 -- 172.21.15.3:0/85666467 learned_addr learned my addr 172.21.15.3:0/85666467 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:38.506 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.505+0000 7f8dff39b640 1 -- 172.21.15.3:0/85666467 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1dbe920 con 0x1f5d8a0 2024-09-22T04:05:38.507 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.505+0000 7f8dff39b640 1 --2- 172.21.15.3:0/85666467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1f5d8a0 0x1ff5690 crc :-1 s=READY pgs=12949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.507 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.505+0000 7f8dfcb96640 1 -- 172.21.15.3:0/85666467 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8de802de50 con 0x1f5d8a0 2024-09-22T04:05:38.508 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.505+0000 7f8dfcb96640 1 -- 172.21.15.3:0/85666467 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8de802d920 con 0x1f5d8a0 2024-09-22T04:05:38.508 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.505+0000 7f8e07253ec0 1 -- 172.21.15.3:0/85666467 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1dbe380 con 0x1f5d8a0 2024-09-22T04:05:38.508 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.505+0000 7f8dfcb96640 1 -- 172.21.15.3:0/85666467 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8de8035c40 con 0x1f5d8a0 2024-09-22T04:05:38.508 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.506+0000 7f8e07253ec0 1 -- 172.21.15.3:0/85666467 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1fe6db0 con 0x1f5d8a0 2024-09-22T04:05:38.508 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.507+0000 7f8dfcb96640 1 -- 172.21.15.3:0/85666467 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f8de8034410 con 0x1f5d8a0 2024-09-22T04:05:38.508 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.507+0000 7f8dfcb96640 1 --2- 172.21.15.3:0/85666467 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8de403d360 0x7f8de403f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.509 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.507+0000 7f8dfcb96640 1 -- 172.21.15.3:0/85666467 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(725..725 src has 590..725) ==== 4207+0+0 (crc 0 0 0) 0x7f8de8032040 con 0x1f5d8a0 2024-09-22T04:05:38.509 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.507+0000 7f8e07253ec0 1 --2- 172.21.15.3:0/85666467 >> v2:172.21.15.123:6806/4206675901 conn(0x1f5e1f0 0x1f56b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.509 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.507+0000 7f8e07253ec0 1 -- 172.21.15.3:0/85666467 --> v2:172.21.15.123:6806/4206675901 -- osd_op(unknown.0.0:1 200.7 200:ff7b1f36:::obj1:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e725) -- 0x1f49e90 con 0x1f5e1f0 2024-09-22T04:05:38.509 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.507+0000 7f8dfeb9a640 1 --2- 172.21.15.3:0/85666467 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8de403d360 0x7f8de403f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.509 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.508+0000 7f8dfeb9a640 1 --2- 172.21.15.3:0/85666467 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8de403d360 0x7f8de403f870 crc :-1 s=READY pgs=3854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.509 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.509+0000 7f8dffb9c640 1 --2- 172.21.15.3:0/85666467 >> v2:172.21.15.123:6806/4206675901 conn(0x1f5e1f0 0x1f56b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:05:38.514 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.513+0000 7f8dffb9c640 1 --2- 172.21.15.3:0/85666467 >> v2:172.21.15.123:6806/4206675901 conn(0x1f5e1f0 0x1f56b90 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.540+0000 7f8dffb9c640 1 -- 172.21.15.3:0/85666467 <== osd.3 v2:172.21.15.123:6806/4206675901 1 ==== osd_op_reply(1 obj1 [writefull 0~12875] v725'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f8df000c430 con 0x1f5e1f0 2024-09-22T04:05:38.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.540+0000 7f8e07253ec0 1 -- 172.21.15.3:0/85666467 >> v2:172.21.15.123:6806/4206675901 conn(0x1f5e1f0 msgr2=0x1f56b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.540+0000 7f8e07253ec0 1 --2- 172.21.15.3:0/85666467 >> v2:172.21.15.123:6806/4206675901 conn(0x1f5e1f0 0x1f56b90 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.540+0000 7f8e07253ec0 1 -- 172.21.15.3:0/85666467 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8de403d360 msgr2=0x7f8de403f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.540+0000 7f8e07253ec0 1 --2- 172.21.15.3:0/85666467 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8de403d360 0x7f8de403f870 crc :-1 s=READY pgs=3854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.540+0000 7f8e07253ec0 1 -- 172.21.15.3:0/85666467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1f5d8a0 msgr2=0x1ff5690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.540+0000 7f8e07253ec0 1 --2- 172.21.15.3:0/85666467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1f5d8a0 0x1ff5690 crc :-1 s=READY pgs=12949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.540+0000 7f8e07253ec0 1 -- 172.21.15.3:0/85666467 shutdown_connections 2024-09-22T04:05:38.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.540+0000 7f8e07253ec0 1 --2- 172.21.15.3:0/85666467 >> v2:172.21.15.123:6806/4206675901 conn(0x1f5e1f0 0x1f56b90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.540+0000 7f8e07253ec0 1 --2- 172.21.15.3:0/85666467 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f8de403d360 0x7f8de403f870 unknown :-1 s=CLOSED pgs=3854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.540+0000 7f8e07253ec0 1 --2- 172.21.15.3:0/85666467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1f5d8a0 0x1ff5690 unknown :-1 s=CLOSED pgs=12949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.542 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.540+0000 7f8e07253ec0 1 -- 172.21.15.3:0/85666467 >> 172.21.15.3:0/85666467 conn(0x1df0770 msgr2=0x1fe3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:38.542 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.540+0000 7f8e07253ec0 1 -- 172.21.15.3:0/85666467 shutdown_connections 2024-09-22T04:05:38.542 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.540+0000 7f8e07253ec0 1 -- 172.21.15.3:0/85666467 wait complete. 2024-09-22T04:05:38.543 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:05:38.543 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p dd65dfbd-105b-4a15-8dc2-717b17ae7efb put obj2 /etc/passwd 2024-09-22T04:05:38.584 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.583+0000 7f55bd033ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.590 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.589+0000 7f55bd033ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.590 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.589+0000 7f55bd033ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.591 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.590+0000 7f55bd033ec0 1 Processor -- start 2024-09-22T04:05:38.591 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.590+0000 7f55bd033ec0 1 -- start start 2024-09-22T04:05:38.593 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.591+0000 7f55bd033ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13e4a70 0x13e2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.593 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.591+0000 7f55bd033ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x12b6520 con 0x13e4a70 2024-09-22T04:05:38.593 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.591+0000 7f55b519b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13e4a70 0x13e2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.593 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.591+0000 7f55b519b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13e4a70 0x13e2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42504/0 (socket says 172.21.15.3:42504) 2024-09-22T04:05:38.593 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.591+0000 7f55b519b640 1 -- 172.21.15.3:0/1242334606 learned_addr learned my addr 172.21.15.3:0/1242334606 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:38.593 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.591+0000 7f55b519b640 1 -- 172.21.15.3:0/1242334606 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1252e10 con 0x13e4a70 2024-09-22T04:05:38.593 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.592+0000 7f55b519b640 1 --2- 172.21.15.3:0/1242334606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13e4a70 0x13e2f80 crc :-1 s=READY pgs=12952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897822c6b687f96e server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.593 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.592+0000 7f55af7fe640 1 -- 172.21.15.3:0/1242334606 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f55a000c490 con 0x13e4a70 2024-09-22T04:05:38.593 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.592+0000 7f55af7fe640 1 -- 172.21.15.3:0/1242334606 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f55a000c5f0 con 0x13e4a70 2024-09-22T04:05:38.593 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.592+0000 7f55af7fe640 1 -- 172.21.15.3:0/1242334606 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f55a000c890 con 0x13e4a70 2024-09-22T04:05:38.594 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.592+0000 7f55bd033ec0 1 -- 172.21.15.3:0/1242334606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13e4a70 msgr2=0x13e2f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.594 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.592+0000 7f55bd033ec0 1 --2- 172.21.15.3:0/1242334606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13e4a70 0x13e2f80 crc :-1 s=READY pgs=12952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.594 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.592+0000 7f55bd033ec0 1 -- 172.21.15.3:0/1242334606 shutdown_connections 2024-09-22T04:05:38.594 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.593+0000 7f55bd033ec0 1 --2- 172.21.15.3:0/1242334606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13e4a70 0x13e2f80 unknown :-1 s=CLOSED pgs=12952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.594 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.593+0000 7f55bd033ec0 1 -- 172.21.15.3:0/1242334606 >> 172.21.15.3:0/1242334606 conn(0x1276770 msgr2=0x13e0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:38.594 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.593+0000 7f55bd033ec0 1 -- 172.21.15.3:0/1242334606 shutdown_connections 2024-09-22T04:05:38.594 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.593+0000 7f55bd033ec0 1 -- 172.21.15.3:0/1242334606 wait complete. 2024-09-22T04:05:38.595 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.594+0000 7f55bd033ec0 1 Processor -- start 2024-09-22T04:05:38.595 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.594+0000 7f55bd033ec0 1 -- start start 2024-09-22T04:05:38.597 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.595+0000 7f55bd033ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13e4a70 0x13cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.597 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.595+0000 7f55bd033ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x12b4280 con 0x13e4a70 2024-09-22T04:05:38.597 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.595+0000 7f55b519b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13e4a70 0x13cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.597 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.595+0000 7f55b519b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13e4a70 0x13cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42510/0 (socket says 172.21.15.3:42510) 2024-09-22T04:05:38.597 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.595+0000 7f55b519b640 1 -- 172.21.15.3:0/3569536911 learned_addr learned my addr 172.21.15.3:0/3569536911 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:38.597 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.596+0000 7f55b519b640 1 -- 172.21.15.3:0/3569536911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1244920 con 0x13e4a70 2024-09-22T04:05:38.597 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.596+0000 7f55b519b640 1 --2- 172.21.15.3:0/3569536911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13e4a70 0x13cca20 crc :-1 s=READY pgs=12953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.596+0000 7f55ae7fc640 1 -- 172.21.15.3:0/3569536911 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f55a00111e0 con 0x13e4a70 2024-09-22T04:05:38.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.596+0000 7f55ae7fc640 1 -- 172.21.15.3:0/3569536911 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f55a00117a0 con 0x13e4a70 2024-09-22T04:05:38.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.596+0000 7f55bd033ec0 1 -- 172.21.15.3:0/3569536911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1244380 con 0x13e4a70 2024-09-22T04:05:38.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.596+0000 7f55ae7fc640 1 -- 172.21.15.3:0/3569536911 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f55a0011910 con 0x13e4a70 2024-09-22T04:05:38.598 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.596+0000 7f55bd033ec0 1 -- 172.21.15.3:0/3569536911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x134f800 con 0x13e4a70 2024-09-22T04:05:38.599 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.597+0000 7f55ae7fc640 1 -- 172.21.15.3:0/3569536911 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f55a0011ac0 con 0x13e4a70 2024-09-22T04:05:38.599 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.597+0000 7f55ae7fc640 1 --2- 172.21.15.3:0/3569536911 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f559c03d360 0x7f559c03f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.599 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.597+0000 7f55ae7fc640 1 -- 172.21.15.3:0/3569536911 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(725..725 src has 590..725) ==== 4207+0+0 (crc 0 0 0) 0x7f55a0050540 con 0x13e4a70 2024-09-22T04:05:38.599 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.598+0000 7f55bd033ec0 1 --2- 172.21.15.3:0/3569536911 >> v2:172.21.15.123:6806/4206675901 conn(0x13e7910 0x13dd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.599 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.598+0000 7f55bd033ec0 1 -- 172.21.15.3:0/3569536911 --> v2:172.21.15.123:6806/4206675901 -- osd_op(unknown.0.0:1 200.8 200:104778fc:::obj2:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e725) -- 0x13cf540 con 0x13e7910 2024-09-22T04:05:38.599 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.598+0000 7f55b499a640 1 --2- 172.21.15.3:0/3569536911 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f559c03d360 0x7f559c03f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.600 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.598+0000 7f55b499a640 1 --2- 172.21.15.3:0/3569536911 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f559c03d360 0x7f559c03f870 crc :-1 s=READY pgs=3855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.600 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.599+0000 7f55b599c640 1 --2- 172.21.15.3:0/3569536911 >> v2:172.21.15.123:6806/4206675901 conn(0x13e7910 0x13dd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:05:38.604 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.603+0000 7f55b599c640 1 --2- 172.21.15.3:0/3569536911 >> v2:172.21.15.123:6806/4206675901 conn(0x13e7910 0x13dd7b0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.629 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.628+0000 7f55b599c640 1 -- 172.21.15.3:0/3569536911 <== osd.3 v2:172.21.15.123:6806/4206675901 1 ==== osd_op_reply(1 obj2 [writefull 0~12875] v725'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f55a800a430 con 0x13e7910 2024-09-22T04:05:38.630 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.628+0000 7f55bd033ec0 1 -- 172.21.15.3:0/3569536911 >> v2:172.21.15.123:6806/4206675901 conn(0x13e7910 msgr2=0x13dd7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.630 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.628+0000 7f55bd033ec0 1 --2- 172.21.15.3:0/3569536911 >> v2:172.21.15.123:6806/4206675901 conn(0x13e7910 0x13dd7b0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.630 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.628+0000 7f55bd033ec0 1 -- 172.21.15.3:0/3569536911 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f559c03d360 msgr2=0x7f559c03f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.630 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.628+0000 7f55bd033ec0 1 --2- 172.21.15.3:0/3569536911 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f559c03d360 0x7f559c03f870 crc :-1 s=READY pgs=3855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.630 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.628+0000 7f55bd033ec0 1 -- 172.21.15.3:0/3569536911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13e4a70 msgr2=0x13cca20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.630 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.628+0000 7f55bd033ec0 1 --2- 172.21.15.3:0/3569536911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13e4a70 0x13cca20 crc :-1 s=READY pgs=12953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.630 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.628+0000 7f55bd033ec0 1 -- 172.21.15.3:0/3569536911 shutdown_connections 2024-09-22T04:05:38.630 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.628+0000 7f55bd033ec0 1 --2- 172.21.15.3:0/3569536911 >> v2:172.21.15.123:6806/4206675901 conn(0x13e7910 0x13dd7b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.630 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.628+0000 7f55bd033ec0 1 --2- 172.21.15.3:0/3569536911 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f559c03d360 0x7f559c03f870 unknown :-1 s=CLOSED pgs=3855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.630 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.628+0000 7f55bd033ec0 1 --2- 172.21.15.3:0/3569536911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x13e4a70 0x13cca20 unknown :-1 s=CLOSED pgs=12953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.630 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.628+0000 7f55bd033ec0 1 -- 172.21.15.3:0/3569536911 >> 172.21.15.3:0/3569536911 conn(0x1276770 msgr2=0x13e4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:38.630 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.628+0000 7f55bd033ec0 1 -- 172.21.15.3:0/3569536911 shutdown_connections 2024-09-22T04:05:38.631 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.628+0000 7f55bd033ec0 1 -- 172.21.15.3:0/3569536911 wait complete. 2024-09-22T04:05:38.632 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:05:38.632 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p dd65dfbd-105b-4a15-8dc2-717b17ae7efb put obj3 /etc/passwd 2024-09-22T04:05:38.670 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.669+0000 7f45368f3ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.678 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.675+0000 7f45368f3ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.678 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.675+0000 7f45368f3ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.678 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.676+0000 7f45368f3ec0 1 Processor -- start 2024-09-22T04:05:38.678 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.676+0000 7f45368f3ec0 1 -- start start 2024-09-22T04:05:38.679 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.677+0000 7f45368f3ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xa6cf80 0xaeb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.679 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.677+0000 7f45368f3ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x9d1520 con 0xa6cf80 2024-09-22T04:05:38.679 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.678+0000 7f452ec3f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xa6cf80 0xaeb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.679 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.678+0000 7f452ec3f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xa6cf80 0xaeb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42528/0 (socket says 172.21.15.3:42528) 2024-09-22T04:05:38.679 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.678+0000 7f452ec3f640 1 -- 172.21.15.3:0/1517049678 learned_addr learned my addr 172.21.15.3:0/1517049678 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:38.679 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.678+0000 7f452ec3f640 1 -- 172.21.15.3:0/1517049678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x96de10 con 0xa6cf80 2024-09-22T04:05:38.679 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.678+0000 7f452ec3f640 1 --2- 172.21.15.3:0/1517049678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xa6cf80 0xaeb6b0 crc :-1 s=READY pgs=12956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11381abb2fe5be5e server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.680 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.678+0000 7f452d43c640 1 -- 172.21.15.3:0/1517049678 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f451802db90 con 0xa6cf80 2024-09-22T04:05:38.680 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.678+0000 7f452d43c640 1 -- 172.21.15.3:0/1517049678 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4518037370 con 0xa6cf80 2024-09-22T04:05:38.680 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.678+0000 7f452d43c640 1 -- 172.21.15.3:0/1517049678 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f45180374e0 con 0xa6cf80 2024-09-22T04:05:38.680 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.679+0000 7f45368f3ec0 1 -- 172.21.15.3:0/1517049678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xa6cf80 msgr2=0xaeb6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.681 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.679+0000 7f45368f3ec0 1 --2- 172.21.15.3:0/1517049678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xa6cf80 0xaeb6b0 crc :-1 s=READY pgs=12956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.681 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.679+0000 7f45368f3ec0 1 -- 172.21.15.3:0/1517049678 shutdown_connections 2024-09-22T04:05:38.681 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.679+0000 7f45368f3ec0 1 --2- 172.21.15.3:0/1517049678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xa6cf80 0xaeb6b0 unknown :-1 s=CLOSED pgs=12956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.681 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.679+0000 7f45368f3ec0 1 -- 172.21.15.3:0/1517049678 >> 172.21.15.3:0/1517049678 conn(0x991770 msgr2=0xa67790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:38.681 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.679+0000 7f45368f3ec0 1 -- 172.21.15.3:0/1517049678 shutdown_connections 2024-09-22T04:05:38.681 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.679+0000 7f45368f3ec0 1 -- 172.21.15.3:0/1517049678 wait complete. 2024-09-22T04:05:38.681 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.680+0000 7f45368f3ec0 1 Processor -- start 2024-09-22T04:05:38.681 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.680+0000 7f45368f3ec0 1 -- start start 2024-09-22T04:05:38.683 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.681+0000 7f45368f3ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xa6cf80 0xaeee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.683 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.681+0000 7f45368f3ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x9cf280 con 0xa6cf80 2024-09-22T04:05:38.683 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.681+0000 7f452ec3f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xa6cf80 0xaeee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.683 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.681+0000 7f452ec3f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xa6cf80 0xaeee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42544/0 (socket says 172.21.15.3:42544) 2024-09-22T04:05:38.683 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.681+0000 7f452ec3f640 1 -- 172.21.15.3:0/948810162 learned_addr learned my addr 172.21.15.3:0/948810162 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:38.683 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.682+0000 7f452ec3f640 1 -- 172.21.15.3:0/948810162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x95f920 con 0xa6cf80 2024-09-22T04:05:38.683 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.682+0000 7f452ec3f640 1 --2- 172.21.15.3:0/948810162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xa6cf80 0xaeee50 crc :-1 s=READY pgs=12957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.684 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.682+0000 7f4527fff640 1 -- 172.21.15.3:0/948810162 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f451803e040 con 0xa6cf80 2024-09-22T04:05:38.684 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.682+0000 7f4527fff640 1 -- 172.21.15.3:0/948810162 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f45180057c0 con 0xa6cf80 2024-09-22T04:05:38.684 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.682+0000 7f45368f3ec0 1 -- 172.21.15.3:0/948810162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x95f380 con 0xa6cf80 2024-09-22T04:05:38.684 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.682+0000 7f4527fff640 1 -- 172.21.15.3:0/948810162 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f451802db90 con 0xa6cf80 2024-09-22T04:05:38.684 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.682+0000 7f45368f3ec0 1 -- 172.21.15.3:0/948810162 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0xaf1ed0 con 0xa6cf80 2024-09-22T04:05:38.685 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.683+0000 7f4527fff640 1 -- 172.21.15.3:0/948810162 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f4518041040 con 0xa6cf80 2024-09-22T04:05:38.685 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.683+0000 7f4527fff640 1 --2- 172.21.15.3:0/948810162 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f451403d310 0x7f451403f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.685 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.683+0000 7f4527fff640 1 -- 172.21.15.3:0/948810162 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(725..725 src has 590..725) ==== 4207+0+0 (crc 0 0 0) 0x7f4518032040 con 0xa6cf80 2024-09-22T04:05:38.685 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.684+0000 7f45368f3ec0 1 --2- 172.21.15.3:0/948810162 >> v2:172.21.15.3:6802/2784566484 conn(0xa6dcd0 0xf98a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.685 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.684+0000 7f45368f3ec0 1 -- 172.21.15.3:0/948810162 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:1 200.1 200:8dd16f86:::obj3:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e725) -- 0xaf2750 con 0xa6dcd0 2024-09-22T04:05:38.685 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.684+0000 7f452e43e640 1 --2- 172.21.15.3:0/948810162 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f451403d310 0x7f451403f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.685 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.684+0000 7f452e43e640 1 --2- 172.21.15.3:0/948810162 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f451403d310 0x7f451403f820 crc :-1 s=READY pgs=3856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.686 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.685+0000 7f452f440640 1 --2- 172.21.15.3:0/948810162 >> v2:172.21.15.3:6802/2784566484 conn(0xa6dcd0 0xf98a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:05:38.689 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.688+0000 7f452f440640 1 --2- 172.21.15.3:0/948810162 >> v2:172.21.15.3:6802/2784566484 conn(0xa6dcd0 0xf98a60 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.712+0000 7f452f440640 1 -- 172.21.15.3:0/948810162 <== osd.1 v2:172.21.15.3:6802/2784566484 1 ==== osd_op_reply(1 obj3 [writefull 0~12875] v725'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f452000a430 con 0xa6dcd0 2024-09-22T04:05:38.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.712+0000 7f45368f3ec0 1 -- 172.21.15.3:0/948810162 >> v2:172.21.15.3:6802/2784566484 conn(0xa6dcd0 msgr2=0xf98a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.712+0000 7f45368f3ec0 1 --2- 172.21.15.3:0/948810162 >> v2:172.21.15.3:6802/2784566484 conn(0xa6dcd0 0xf98a60 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.712+0000 7f45368f3ec0 1 -- 172.21.15.3:0/948810162 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f451403d310 msgr2=0x7f451403f820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.712+0000 7f45368f3ec0 1 --2- 172.21.15.3:0/948810162 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f451403d310 0x7f451403f820 crc :-1 s=READY pgs=3856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.712+0000 7f45368f3ec0 1 -- 172.21.15.3:0/948810162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xa6cf80 msgr2=0xaeee50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.712+0000 7f45368f3ec0 1 --2- 172.21.15.3:0/948810162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xa6cf80 0xaeee50 crc :-1 s=READY pgs=12957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.712+0000 7f45368f3ec0 1 -- 172.21.15.3:0/948810162 shutdown_connections 2024-09-22T04:05:38.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.712+0000 7f45368f3ec0 1 --2- 172.21.15.3:0/948810162 >> v2:172.21.15.3:6802/2784566484 conn(0xa6dcd0 0xf98a60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.712+0000 7f45368f3ec0 1 --2- 172.21.15.3:0/948810162 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f451403d310 0x7f451403f820 unknown :-1 s=CLOSED pgs=3856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.715 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.712+0000 7f45368f3ec0 1 --2- 172.21.15.3:0/948810162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xa6cf80 0xaeee50 unknown :-1 s=CLOSED pgs=12957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.715 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.712+0000 7f45368f3ec0 1 -- 172.21.15.3:0/948810162 >> 172.21.15.3:0/948810162 conn(0x991770 msgr2=0xa71710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:38.715 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.712+0000 7f45368f3ec0 1 -- 172.21.15.3:0/948810162 shutdown_connections 2024-09-22T04:05:38.715 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.712+0000 7f45368f3ec0 1 -- 172.21.15.3:0/948810162 wait complete. 2024-09-22T04:05:38.716 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:05:38.716 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p dd65dfbd-105b-4a15-8dc2-717b17ae7efb put obj4 /etc/passwd 2024-09-22T04:05:38.754 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.752+0000 7f8077743ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.759 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.758+0000 7f8077743ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.759 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.758+0000 7f8077743ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.760 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.759+0000 7f8077743ec0 1 Processor -- start 2024-09-22T04:05:38.760 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.759+0000 7f8077743ec0 1 -- start start 2024-09-22T04:05:38.763 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.761+0000 7f8077743ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcc1150 0xde0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.763 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.761+0000 7f8077743ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0xc25520 con 0xcc1150 2024-09-22T04:05:38.763 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.761+0000 7f806b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcc1150 0xde0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.763 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.761+0000 7f806b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcc1150 0xde0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42566/0 (socket says 172.21.15.3:42566) 2024-09-22T04:05:38.763 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.761+0000 7f806b7fe640 1 -- 172.21.15.3:0/111240750 learned_addr learned my addr 172.21.15.3:0/111240750 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:38.763 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.761+0000 7f806b7fe640 1 -- 172.21.15.3:0/111240750 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xbc1e10 con 0xcc1150 2024-09-22T04:05:38.763 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.761+0000 7f806b7fe640 1 --2- 172.21.15.3:0/111240750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcc1150 0xde0e10 crc :-1 s=READY pgs=12960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81c85d110bc0f94 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.763 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.761+0000 7f8069ffb640 1 -- 172.21.15.3:0/111240750 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f80580061e0 con 0xcc1150 2024-09-22T04:05:38.763 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.761+0000 7f8069ffb640 1 -- 172.21.15.3:0/111240750 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f80580019e0 con 0xcc1150 2024-09-22T04:05:38.763 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.761+0000 7f8069ffb640 1 -- 172.21.15.3:0/111240750 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8058006340 con 0xcc1150 2024-09-22T04:05:38.764 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.762+0000 7f8077743ec0 1 -- 172.21.15.3:0/111240750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcc1150 msgr2=0xde0e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.764 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.762+0000 7f8077743ec0 1 --2- 172.21.15.3:0/111240750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcc1150 0xde0e10 crc :-1 s=READY pgs=12960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.764 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.762+0000 7f8077743ec0 1 -- 172.21.15.3:0/111240750 shutdown_connections 2024-09-22T04:05:38.764 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.762+0000 7f8077743ec0 1 --2- 172.21.15.3:0/111240750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcc1150 0xde0e10 unknown :-1 s=CLOSED pgs=12960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.764 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.762+0000 7f8077743ec0 1 -- 172.21.15.3:0/111240750 >> 172.21.15.3:0/111240750 conn(0xbe5770 msgr2=0xcbb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:38.765 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.762+0000 7f8077743ec0 1 -- 172.21.15.3:0/111240750 shutdown_connections 2024-09-22T04:05:38.765 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.762+0000 7f8077743ec0 1 -- 172.21.15.3:0/111240750 wait complete. 2024-09-22T04:05:38.765 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.763+0000 7f8077743ec0 1 Processor -- start 2024-09-22T04:05:38.765 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.763+0000 7f8077743ec0 1 -- start start 2024-09-22T04:05:38.766 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.765+0000 7f8077743ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcc1150 0xd50dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.767 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.765+0000 7f8077743ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0xc23280 con 0xcc1150 2024-09-22T04:05:38.767 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.765+0000 7f806b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcc1150 0xd50dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.767 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.765+0000 7f806b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcc1150 0xd50dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42582/0 (socket says 172.21.15.3:42582) 2024-09-22T04:05:38.767 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.765+0000 7f806b7fe640 1 -- 172.21.15.3:0/2124638337 learned_addr learned my addr 172.21.15.3:0/2124638337 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:38.767 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.765+0000 7f806b7fe640 1 -- 172.21.15.3:0/2124638337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xbb3920 con 0xcc1150 2024-09-22T04:05:38.767 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.765+0000 7f806b7fe640 1 --2- 172.21.15.3:0/2124638337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcc1150 0xd50dc0 crc :-1 s=READY pgs=12961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.767 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.765+0000 7f8068ff9640 1 -- 172.21.15.3:0/2124638337 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8058004b20 con 0xcc1150 2024-09-22T04:05:38.767 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.765+0000 7f8068ff9640 1 -- 172.21.15.3:0/2124638337 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8058004c80 con 0xcc1150 2024-09-22T04:05:38.767 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.765+0000 7f8068ff9640 1 -- 172.21.15.3:0/2124638337 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8058012330 con 0xcc1150 2024-09-22T04:05:38.767 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.765+0000 7f8077743ec0 1 -- 172.21.15.3:0/2124638337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0xbb3380 con 0xcc1150 2024-09-22T04:05:38.767 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.765+0000 7f8077743ec0 1 -- 172.21.15.3:0/2124638337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0xddd380 con 0xcc1150 2024-09-22T04:05:38.768 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.766+0000 7f8068ff9640 1 -- 172.21.15.3:0/2124638337 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f8058012530 con 0xcc1150 2024-09-22T04:05:38.768 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.767+0000 7f8068ff9640 1 --2- 172.21.15.3:0/2124638337 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f804803d310 0x7f804803f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.768 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.767+0000 7f8068ff9640 1 -- 172.21.15.3:0/2124638337 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(725..725 src has 590..725) ==== 4207+0+0 (crc 0 0 0) 0x7f8058050180 con 0xcc1150 2024-09-22T04:05:38.768 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.767+0000 7f8077743ec0 1 --2- 172.21.15.3:0/2124638337 >> v2:172.21.15.123:6806/4206675901 conn(0xcc1cd0 0xdda0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.768 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.767+0000 7f8077743ec0 1 -- 172.21.15.3:0/2124638337 --> v2:172.21.15.123:6806/4206675901 -- osd_op(unknown.0.0:1 200.0 200:0ee9ae15:::obj4:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e725) -- 0xddda50 con 0xcc1cd0 2024-09-22T04:05:38.768 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.767+0000 7f806affd640 1 --2- 172.21.15.3:0/2124638337 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f804803d310 0x7f804803f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.769 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.768+0000 7f806affd640 1 --2- 172.21.15.3:0/2124638337 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f804803d310 0x7f804803f820 crc :-1 s=READY pgs=3857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.769 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.768+0000 7f806bfff640 1 --2- 172.21.15.3:0/2124638337 >> v2:172.21.15.123:6806/4206675901 conn(0xcc1cd0 0xdda0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:05:38.771 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.770+0000 7f806bfff640 1 --2- 172.21.15.3:0/2124638337 >> v2:172.21.15.123:6806/4206675901 conn(0xcc1cd0 0xdda0d0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.796+0000 7f806bfff640 1 -- 172.21.15.3:0/2124638337 <== osd.3 v2:172.21.15.123:6806/4206675901 1 ==== osd_op_reply(1 obj4 [writefull 0~12875] v725'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f806000c430 con 0xcc1cd0 2024-09-22T04:05:38.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.796+0000 7f8077743ec0 1 -- 172.21.15.3:0/2124638337 >> v2:172.21.15.123:6806/4206675901 conn(0xcc1cd0 msgr2=0xdda0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.796+0000 7f8077743ec0 1 --2- 172.21.15.3:0/2124638337 >> v2:172.21.15.123:6806/4206675901 conn(0xcc1cd0 0xdda0d0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.796+0000 7f8077743ec0 1 -- 172.21.15.3:0/2124638337 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f804803d310 msgr2=0x7f804803f820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.796+0000 7f8077743ec0 1 --2- 172.21.15.3:0/2124638337 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f804803d310 0x7f804803f820 crc :-1 s=READY pgs=3857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.796+0000 7f8077743ec0 1 -- 172.21.15.3:0/2124638337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcc1150 msgr2=0xd50dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.796+0000 7f8077743ec0 1 --2- 172.21.15.3:0/2124638337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcc1150 0xd50dc0 crc :-1 s=READY pgs=12961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.796+0000 7f8077743ec0 1 -- 172.21.15.3:0/2124638337 shutdown_connections 2024-09-22T04:05:38.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.796+0000 7f8077743ec0 1 --2- 172.21.15.3:0/2124638337 >> v2:172.21.15.123:6806/4206675901 conn(0xcc1cd0 0xdda0d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.796+0000 7f8077743ec0 1 --2- 172.21.15.3:0/2124638337 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f804803d310 0x7f804803f820 unknown :-1 s=CLOSED pgs=3857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.796+0000 7f8077743ec0 1 --2- 172.21.15.3:0/2124638337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xcc1150 0xd50dc0 unknown :-1 s=CLOSED pgs=12961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.797 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.796+0000 7f8077743ec0 1 -- 172.21.15.3:0/2124638337 >> 172.21.15.3:0/2124638337 conn(0xbe5770 msgr2=0xcc5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:38.798 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.796+0000 7f8077743ec0 1 -- 172.21.15.3:0/2124638337 shutdown_connections 2024-09-22T04:05:38.798 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.796+0000 7f8077743ec0 1 -- 172.21.15.3:0/2124638337 wait complete. 2024-09-22T04:05:38.799 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:05:38.799 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p dd65dfbd-105b-4a15-8dc2-717b17ae7efb put obj5 /etc/passwd 2024-09-22T04:05:38.836 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.835+0000 7f22cacfeec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.842 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.841+0000 7f22cacfeec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.842 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.841+0000 7f22cacfeec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.843 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.842+0000 7f22cacfeec0 1 Processor -- start 2024-09-22T04:05:38.843 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.842+0000 7f22cacfeec0 1 -- start start 2024-09-22T04:05:38.845 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.843+0000 7f22cacfeec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7c07f0 0x7ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.845 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.843+0000 7f22cacfeec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x693520 con 0x7c07f0 2024-09-22T04:05:38.845 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.843+0000 7f22c319b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7c07f0 0x7ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.845 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.843+0000 7f22c319b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7c07f0 0x7ab6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42600/0 (socket says 172.21.15.3:42600) 2024-09-22T04:05:38.845 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.843+0000 7f22c319b640 1 -- 172.21.15.3:0/2187055587 learned_addr learned my addr 172.21.15.3:0/2187055587 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:38.846 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.843+0000 7f22c319b640 1 -- 172.21.15.3:0/2187055587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x62fe10 con 0x7c07f0 2024-09-22T04:05:38.846 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.844+0000 7f22c319b640 1 --2- 172.21.15.3:0/2187055587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7c07f0 0x7ab6a0 crc :-1 s=READY pgs=12964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a70e08ec7a1357 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.846 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.844+0000 7f22c1998640 1 -- 172.21.15.3:0/2187055587 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f22ac036430 con 0x7c07f0 2024-09-22T04:05:38.846 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.844+0000 7f22c1998640 1 -- 172.21.15.3:0/2187055587 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f22ac036590 con 0x7c07f0 2024-09-22T04:05:38.846 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.844+0000 7f22c1998640 1 -- 172.21.15.3:0/2187055587 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f22ac036830 con 0x7c07f0 2024-09-22T04:05:38.847 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.844+0000 7f22cacfeec0 1 -- 172.21.15.3:0/2187055587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7c07f0 msgr2=0x7ab6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.847 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.844+0000 7f22cacfeec0 1 --2- 172.21.15.3:0/2187055587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7c07f0 0x7ab6a0 crc :-1 s=READY pgs=12964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.847 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.844+0000 7f22cacfeec0 1 -- 172.21.15.3:0/2187055587 shutdown_connections 2024-09-22T04:05:38.847 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.844+0000 7f22cacfeec0 1 --2- 172.21.15.3:0/2187055587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7c07f0 0x7ab6a0 unknown :-1 s=CLOSED pgs=12964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.847 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.844+0000 7f22cacfeec0 1 -- 172.21.15.3:0/2187055587 >> 172.21.15.3:0/2187055587 conn(0x653770 msgr2=0x728690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:38.847 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.844+0000 7f22cacfeec0 1 -- 172.21.15.3:0/2187055587 shutdown_connections 2024-09-22T04:05:38.847 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.845+0000 7f22cacfeec0 1 -- 172.21.15.3:0/2187055587 wait complete. 2024-09-22T04:05:38.847 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.845+0000 7f22cacfeec0 1 Processor -- start 2024-09-22T04:05:38.847 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.846+0000 7f22cacfeec0 1 -- start start 2024-09-22T04:05:38.848 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.847+0000 7f22cacfeec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7c07f0 0x7b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.848 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.847+0000 7f22cacfeec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x691280 con 0x7c07f0 2024-09-22T04:05:38.848 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.847+0000 7f22c319b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7c07f0 0x7b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.849 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.847+0000 7f22c319b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7c07f0 0x7b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42608/0 (socket says 172.21.15.3:42608) 2024-09-22T04:05:38.849 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.847+0000 7f22c319b640 1 -- 172.21.15.3:0/2775408963 learned_addr learned my addr 172.21.15.3:0/2775408963 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:38.849 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.847+0000 7f22c319b640 1 -- 172.21.15.3:0/2775408963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x621920 con 0x7c07f0 2024-09-22T04:05:38.849 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.847+0000 7f22c319b640 1 --2- 172.21.15.3:0/2775408963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7c07f0 0x7b1d30 crc :-1 s=READY pgs=12965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.849 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.848+0000 7f22c0996640 1 -- 172.21.15.3:0/2775408963 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f22ac036700 con 0x7c07f0 2024-09-22T04:05:38.849 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.848+0000 7f22c0996640 1 -- 172.21.15.3:0/2775408963 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f22ac0353c0 con 0x7c07f0 2024-09-22T04:05:38.850 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.848+0000 7f22cacfeec0 1 -- 172.21.15.3:0/2775408963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x621380 con 0x7c07f0 2024-09-22T04:05:38.850 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.848+0000 7f22c0996640 1 -- 172.21.15.3:0/2775408963 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f22ac035530 con 0x7c07f0 2024-09-22T04:05:38.850 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.848+0000 7f22cacfeec0 1 -- 172.21.15.3:0/2775408963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7b4fc0 con 0x7c07f0 2024-09-22T04:05:38.850 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.849+0000 7f22c0996640 1 -- 172.21.15.3:0/2775408963 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f22ac003040 con 0x7c07f0 2024-09-22T04:05:38.851 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.849+0000 7f22c0996640 1 --2- 172.21.15.3:0/2775408963 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f22a803d360 0x7f22a803f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.851 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.849+0000 7f22c0996640 1 -- 172.21.15.3:0/2775408963 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(725..725 src has 590..725) ==== 4207+0+0 (crc 0 0 0) 0x7f22ac03b040 con 0x7c07f0 2024-09-22T04:05:38.851 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.849+0000 7f22cacfeec0 1 --2- 172.21.15.3:0/2775408963 >> v2:172.21.15.123:6806/4206675901 conn(0x7c1140 0x858840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.851 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.849+0000 7f22cacfeec0 1 -- 172.21.15.3:0/2775408963 --> v2:172.21.15.123:6806/4206675901 -- osd_op(unknown.0.0:1 200.3 200:c52c9666:::obj5:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e725) -- 0x7b5600 con 0x7c1140 2024-09-22T04:05:38.851 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.849+0000 7f22c299a640 1 --2- 172.21.15.3:0/2775408963 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f22a803d360 0x7f22a803f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.851 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.850+0000 7f22c299a640 1 --2- 172.21.15.3:0/2775408963 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f22a803d360 0x7f22a803f870 crc :-1 s=READY pgs=3858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.851 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.850+0000 7f22c399c640 1 --2- 172.21.15.3:0/2775408963 >> v2:172.21.15.123:6806/4206675901 conn(0x7c1140 0x858840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:05:38.855 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.854+0000 7f22c399c640 1 --2- 172.21.15.3:0/2775408963 >> v2:172.21.15.123:6806/4206675901 conn(0x7c1140 0x858840 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.880 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.878+0000 7f22c399c640 1 -- 172.21.15.3:0/2775408963 <== osd.3 v2:172.21.15.123:6806/4206675901 1 ==== osd_op_reply(1 obj5 [writefull 0~12875] v725'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f22b400c430 con 0x7c1140 2024-09-22T04:05:38.880 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.878+0000 7f22cacfeec0 1 -- 172.21.15.3:0/2775408963 >> v2:172.21.15.123:6806/4206675901 conn(0x7c1140 msgr2=0x858840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.880 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.878+0000 7f22cacfeec0 1 --2- 172.21.15.3:0/2775408963 >> v2:172.21.15.123:6806/4206675901 conn(0x7c1140 0x858840 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.880 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.878+0000 7f22cacfeec0 1 -- 172.21.15.3:0/2775408963 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f22a803d360 msgr2=0x7f22a803f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.880 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.878+0000 7f22cacfeec0 1 --2- 172.21.15.3:0/2775408963 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f22a803d360 0x7f22a803f870 crc :-1 s=READY pgs=3858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.880 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.878+0000 7f22cacfeec0 1 -- 172.21.15.3:0/2775408963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7c07f0 msgr2=0x7b1d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.880 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.878+0000 7f22cacfeec0 1 --2- 172.21.15.3:0/2775408963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7c07f0 0x7b1d30 crc :-1 s=READY pgs=12965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.880 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.878+0000 7f22cacfeec0 1 -- 172.21.15.3:0/2775408963 shutdown_connections 2024-09-22T04:05:38.880 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.878+0000 7f22cacfeec0 1 --2- 172.21.15.3:0/2775408963 >> v2:172.21.15.123:6806/4206675901 conn(0x7c1140 0x858840 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.880 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.878+0000 7f22cacfeec0 1 --2- 172.21.15.3:0/2775408963 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f22a803d360 0x7f22a803f870 unknown :-1 s=CLOSED pgs=3858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.880 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.878+0000 7f22cacfeec0 1 --2- 172.21.15.3:0/2775408963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7c07f0 0x7b1d30 unknown :-1 s=CLOSED pgs=12965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.881 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.878+0000 7f22cacfeec0 1 -- 172.21.15.3:0/2775408963 >> 172.21.15.3:0/2775408963 conn(0x653770 msgr2=0x732d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:38.881 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.878+0000 7f22cacfeec0 1 -- 172.21.15.3:0/2775408963 shutdown_connections 2024-09-22T04:05:38.881 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.878+0000 7f22cacfeec0 1 -- 172.21.15.3:0/2775408963 wait complete. 2024-09-22T04:05:38.882 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:05:38.882 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p dd65dfbd-105b-4a15-8dc2-717b17ae7efb put obj6 /etc/passwd 2024-09-22T04:05:38.919 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.918+0000 7f3f16071ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.925 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.924+0000 7f3f16071ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.925 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.924+0000 7f3f16071ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:38.926 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.925+0000 7f3f16071ec0 1 Processor -- start 2024-09-22T04:05:38.926 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.925+0000 7f3f16071ec0 1 -- start start 2024-09-22T04:05:38.928 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.926+0000 7f3f16071ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1a547f0 0x1adb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.928 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.926+0000 7f3f16071ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x1927520 con 0x1a547f0 2024-09-22T04:05:38.928 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.927+0000 7f3f0e19b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1a547f0 0x1adb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.928 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.927+0000 7f3f0e19b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1a547f0 0x1adb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42628/0 (socket says 172.21.15.3:42628) 2024-09-22T04:05:38.928 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.927+0000 7f3f0e19b640 1 -- 172.21.15.3:0/870636553 learned_addr learned my addr 172.21.15.3:0/870636553 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:38.929 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.927+0000 7f3f0e19b640 1 -- 172.21.15.3:0/870636553 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x18c3e10 con 0x1a547f0 2024-09-22T04:05:38.929 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.927+0000 7f3f0e19b640 1 --2- 172.21.15.3:0/870636553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1a547f0 0x1adb730 crc :-1 s=READY pgs=12968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e213fabb931826bc server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.929 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.927+0000 7f3f0c998640 1 -- 172.21.15.3:0/870636553 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3ef80061e0 con 0x1a547f0 2024-09-22T04:05:38.929 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.927+0000 7f3f0c998640 1 -- 172.21.15.3:0/870636553 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3ef8006340 con 0x1a547f0 2024-09-22T04:05:38.929 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.927+0000 7f3f0c998640 1 -- 172.21.15.3:0/870636553 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3ef800da40 con 0x1a547f0 2024-09-22T04:05:38.930 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.928+0000 7f3f16071ec0 1 -- 172.21.15.3:0/870636553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1a547f0 msgr2=0x1adb730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.930 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.928+0000 7f3f16071ec0 1 --2- 172.21.15.3:0/870636553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1a547f0 0x1adb730 crc :-1 s=READY pgs=12968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.930 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.928+0000 7f3f16071ec0 1 -- 172.21.15.3:0/870636553 shutdown_connections 2024-09-22T04:05:38.930 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.928+0000 7f3f16071ec0 1 --2- 172.21.15.3:0/870636553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1a547f0 0x1adb730 unknown :-1 s=CLOSED pgs=12968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.930 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.928+0000 7f3f16071ec0 1 -- 172.21.15.3:0/870636553 >> 172.21.15.3:0/870636553 conn(0x18e7770 msgr2=0x19bc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:38.930 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.928+0000 7f3f16071ec0 1 -- 172.21.15.3:0/870636553 shutdown_connections 2024-09-22T04:05:38.930 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.928+0000 7f3f16071ec0 1 -- 172.21.15.3:0/870636553 wait complete. 2024-09-22T04:05:38.930 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.929+0000 7f3f16071ec0 1 Processor -- start 2024-09-22T04:05:38.930 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.929+0000 7f3f16071ec0 1 -- start start 2024-09-22T04:05:38.932 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.930+0000 7f3f16071ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1a547f0 0x1a3f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.933 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.930+0000 7f3f16071ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x1925280 con 0x1a547f0 2024-09-22T04:05:38.933 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.931+0000 7f3f0e19b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1a547f0 0x1a3f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.933 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.931+0000 7f3f0e19b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1a547f0 0x1a3f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42634/0 (socket says 172.21.15.3:42634) 2024-09-22T04:05:38.933 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.931+0000 7f3f0e19b640 1 -- 172.21.15.3:0/3887500078 learned_addr learned my addr 172.21.15.3:0/3887500078 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:38.933 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.931+0000 7f3f0e19b640 1 -- 172.21.15.3:0/3887500078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x18b5920 con 0x1a547f0 2024-09-22T04:05:38.933 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.931+0000 7f3f0e19b640 1 --2- 172.21.15.3:0/3887500078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1a547f0 0x1a3f6e0 crc :-1 s=READY pgs=12969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.933 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.931+0000 7f3f06ffd640 1 -- 172.21.15.3:0/3887500078 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3ef8017040 con 0x1a547f0 2024-09-22T04:05:38.933 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.931+0000 7f3f06ffd640 1 -- 172.21.15.3:0/3887500078 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3ef8005ab0 con 0x1a547f0 2024-09-22T04:05:38.933 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.931+0000 7f3f16071ec0 1 -- 172.21.15.3:0/3887500078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x18b5380 con 0x1a547f0 2024-09-22T04:05:38.933 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.931+0000 7f3f06ffd640 1 -- 172.21.15.3:0/3887500078 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3ef8005d50 con 0x1a547f0 2024-09-22T04:05:38.933 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.931+0000 7f3f16071ec0 1 -- 172.21.15.3:0/3887500078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1add300 con 0x1a547f0 2024-09-22T04:05:38.934 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.932+0000 7f3f06ffd640 1 -- 172.21.15.3:0/3887500078 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f3ef8010440 con 0x1a547f0 2024-09-22T04:05:38.934 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.932+0000 7f3f06ffd640 1 --2- 172.21.15.3:0/3887500078 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f3ee803d360 0x7f3ee803f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.934 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.932+0000 7f3f06ffd640 1 -- 172.21.15.3:0/3887500078 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(725..725 src has 590..725) ==== 4207+0+0 (crc 0 0 0) 0x7f3ef8052330 con 0x1a547f0 2024-09-22T04:05:38.935 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.933+0000 7f3f0d99a640 1 --2- 172.21.15.3:0/3887500078 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f3ee803d360 0x7f3ee803f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:38.935 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.933+0000 7f3f16071ec0 1 --2- 172.21.15.3:0/3887500078 >> v2:172.21.15.3:6802/2784566484 conn(0x1a55310 0x1a53110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:38.935 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.933+0000 7f3f16071ec0 1 -- 172.21.15.3:0/3887500078 --> v2:172.21.15.3:6802/2784566484 -- osd_op(unknown.0.0:1 200.4 200:233a42c1:::obj6:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e725) -- 0x1a41fd0 con 0x1a55310 2024-09-22T04:05:38.935 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.933+0000 7f3f0d99a640 1 --2- 172.21.15.3:0/3887500078 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f3ee803d360 0x7f3ee803f870 crc :-1 s=READY pgs=3859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.935 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.933+0000 7f3f0e99c640 1 --2- 172.21.15.3:0/3887500078 >> v2:172.21.15.3:6802/2784566484 conn(0x1a55310 0x1a53110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:05:38.937 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.936+0000 7f3f0e99c640 1 --2- 172.21.15.3:0/3887500078 >> v2:172.21.15.3:6802/2784566484 conn(0x1a55310 0x1a53110 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:38.968 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.966+0000 7f3f0e99c640 1 -- 172.21.15.3:0/3887500078 <== osd.1 v2:172.21.15.3:6802/2784566484 1 ==== osd_op_reply(1 obj6 [writefull 0~12875] v725'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f3f0000c430 con 0x1a55310 2024-09-22T04:05:38.968 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.966+0000 7f3f16071ec0 1 -- 172.21.15.3:0/3887500078 >> v2:172.21.15.3:6802/2784566484 conn(0x1a55310 msgr2=0x1a53110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.968 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.966+0000 7f3f16071ec0 1 --2- 172.21.15.3:0/3887500078 >> v2:172.21.15.3:6802/2784566484 conn(0x1a55310 0x1a53110 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.969 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.967+0000 7f3f16071ec0 1 -- 172.21.15.3:0/3887500078 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f3ee803d360 msgr2=0x7f3ee803f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.969 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.967+0000 7f3f16071ec0 1 --2- 172.21.15.3:0/3887500078 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f3ee803d360 0x7f3ee803f870 crc :-1 s=READY pgs=3859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.969 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.967+0000 7f3f16071ec0 1 -- 172.21.15.3:0/3887500078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1a547f0 msgr2=0x1a3f6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:38.969 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.967+0000 7f3f16071ec0 1 --2- 172.21.15.3:0/3887500078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1a547f0 0x1a3f6e0 crc :-1 s=READY pgs=12969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.969 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.967+0000 7f3f16071ec0 1 -- 172.21.15.3:0/3887500078 shutdown_connections 2024-09-22T04:05:38.969 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.967+0000 7f3f16071ec0 1 --2- 172.21.15.3:0/3887500078 >> v2:172.21.15.3:6802/2784566484 conn(0x1a55310 0x1a53110 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.969 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.967+0000 7f3f16071ec0 1 --2- 172.21.15.3:0/3887500078 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f3ee803d360 0x7f3ee803f870 unknown :-1 s=CLOSED pgs=3859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.969 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.967+0000 7f3f16071ec0 1 --2- 172.21.15.3:0/3887500078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1a547f0 0x1a3f6e0 unknown :-1 s=CLOSED pgs=12969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:38.969 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.967+0000 7f3f16071ec0 1 -- 172.21.15.3:0/3887500078 >> 172.21.15.3:0/3887500078 conn(0x18e7770 msgr2=0x19bc490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:38.969 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.967+0000 7f3f16071ec0 1 -- 172.21.15.3:0/3887500078 shutdown_connections 2024-09-22T04:05:38.969 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:38.967+0000 7f3f16071ec0 1 -- 172.21.15.3:0/3887500078 wait complete. 2024-09-22T04:05:38.970 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:05:38.970 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p dd65dfbd-105b-4a15-8dc2-717b17ae7efb put obj7 /etc/passwd 2024-09-22T04:05:39.008 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.007+0000 7f788d452ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:39.013 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.012+0000 7f788d452ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:39.013 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.012+0000 7f788d452ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:39.014 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.013+0000 7f788d452ec0 1 Processor -- start 2024-09-22T04:05:39.014 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.013+0000 7f788d452ec0 1 -- start start 2024-09-22T04:05:39.016 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.015+0000 7f788d452ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b1a7f0 0x1b056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:39.017 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.015+0000 7f788d452ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x19ed520 con 0x1b1a7f0 2024-09-22T04:05:39.017 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.015+0000 7f788559b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b1a7f0 0x1b056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:39.017 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.015+0000 7f788559b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b1a7f0 0x1b056a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42652/0 (socket says 172.21.15.3:42652) 2024-09-22T04:05:39.017 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.015+0000 7f788559b640 1 -- 172.21.15.3:0/3287924839 learned_addr learned my addr 172.21.15.3:0/3287924839 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:39.017 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.015+0000 7f788559b640 1 -- 172.21.15.3:0/3287924839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1989e10 con 0x1b1a7f0 2024-09-22T04:05:39.017 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.015+0000 7f788559b640 1 --2- 172.21.15.3:0/3287924839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b1a7f0 0x1b056a0 crc :-1 s=READY pgs=12972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27601dfd9149cf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:39.017 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.015+0000 7f787f7fe640 1 -- 172.21.15.3:0/3287924839 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f78700061f0 con 0x1b1a7f0 2024-09-22T04:05:39.017 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.015+0000 7f787f7fe640 1 -- 172.21.15.3:0/3287924839 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f7870001a30 con 0x1b1a7f0 2024-09-22T04:05:39.017 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.015+0000 7f787f7fe640 1 -- 172.21.15.3:0/3287924839 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7870001ba0 con 0x1b1a7f0 2024-09-22T04:05:39.018 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.016+0000 7f788d452ec0 1 -- 172.21.15.3:0/3287924839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b1a7f0 msgr2=0x1b056a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:39.018 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.016+0000 7f788d452ec0 1 --2- 172.21.15.3:0/3287924839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b1a7f0 0x1b056a0 crc :-1 s=READY pgs=12972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.018 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.016+0000 7f788d452ec0 1 -- 172.21.15.3:0/3287924839 shutdown_connections 2024-09-22T04:05:39.018 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.016+0000 7f788d452ec0 1 --2- 172.21.15.3:0/3287924839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b1a7f0 0x1b056a0 unknown :-1 s=CLOSED pgs=12972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.018 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.016+0000 7f788d452ec0 1 -- 172.21.15.3:0/3287924839 >> 172.21.15.3:0/3287924839 conn(0x19ad770 msgr2=0x1a82690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:39.018 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.016+0000 7f788d452ec0 1 -- 172.21.15.3:0/3287924839 shutdown_connections 2024-09-22T04:05:39.018 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.016+0000 7f788d452ec0 1 -- 172.21.15.3:0/3287924839 wait complete. 2024-09-22T04:05:39.019 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.017+0000 7f788d452ec0 1 Processor -- start 2024-09-22T04:05:39.019 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.017+0000 7f788d452ec0 1 -- start start 2024-09-22T04:05:39.020 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.019+0000 7f788d452ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b1a7f0 0x1b07e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:39.021 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.019+0000 7f788d452ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x19eb280 con 0x1b1a7f0 2024-09-22T04:05:39.021 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.019+0000 7f788559b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b1a7f0 0x1b07e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:39.021 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.019+0000 7f788559b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b1a7f0 0x1b07e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42658/0 (socket says 172.21.15.3:42658) 2024-09-22T04:05:39.021 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.019+0000 7f788559b640 1 -- 172.21.15.3:0/2702358823 learned_addr learned my addr 172.21.15.3:0/2702358823 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:39.021 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.019+0000 7f788559b640 1 -- 172.21.15.3:0/2702358823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x197b920 con 0x1b1a7f0 2024-09-22T04:05:39.021 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.019+0000 7f788559b640 1 --2- 172.21.15.3:0/2702358823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b1a7f0 0x1b07e10 crc :-1 s=READY pgs=12973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:39.021 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.019+0000 7f787e7fc640 1 -- 172.21.15.3:0/2702358823 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7870011e80 con 0x1b1a7f0 2024-09-22T04:05:39.021 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.019+0000 7f787e7fc640 1 -- 172.21.15.3:0/2702358823 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f78700060c0 con 0x1b1a7f0 2024-09-22T04:05:39.021 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.019+0000 7f788d452ec0 1 -- 172.21.15.3:0/2702358823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x197b380 con 0x1b1a7f0 2024-09-22T04:05:39.021 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.019+0000 7f787e7fc640 1 -- 172.21.15.3:0/2702358823 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f7870003450 con 0x1b1a7f0 2024-09-22T04:05:39.021 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.019+0000 7f788d452ec0 1 -- 172.21.15.3:0/2702358823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1b19ca0 con 0x1b1a7f0 2024-09-22T04:05:39.022 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.020+0000 7f787e7fc640 1 -- 172.21.15.3:0/2702358823 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f7870003600 con 0x1b1a7f0 2024-09-22T04:05:39.022 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.021+0000 7f787e7fc640 1 --2- 172.21.15.3:0/2702358823 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f786c03d360 0x7f786c03f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:39.022 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.021+0000 7f787e7fc640 1 -- 172.21.15.3:0/2702358823 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(725..725 src has 590..725) ==== 4207+0+0 (crc 0 0 0) 0x7f7870052490 con 0x1b1a7f0 2024-09-22T04:05:39.022 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.021+0000 7f788d452ec0 1 --2- 172.21.15.3:0/2702358823 >> v2:172.21.15.3:6801/2889521405 conn(0x1b1b310 0x1bae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:39.023 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.021+0000 7f788d452ec0 1 -- 172.21.15.3:0/2702358823 --> v2:172.21.15.3:6801/2889521405 -- osd_op(unknown.0.0:1 200.9 200:9a09113a:::obj7:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e725) -- 0x1b193a0 con 0x1b1b310 2024-09-22T04:05:39.023 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.021+0000 7f7884d9a640 1 --2- 172.21.15.3:0/2702358823 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f786c03d360 0x7f786c03f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:39.023 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.021+0000 7f7884d9a640 1 --2- 172.21.15.3:0/2702358823 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f786c03d360 0x7f786c03f870 crc :-1 s=READY pgs=3860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:39.023 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.022+0000 7f7885d9c640 1 --2- 172.21.15.3:0/2702358823 >> v2:172.21.15.3:6801/2889521405 conn(0x1b1b310 0x1bae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:05:39.026 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.025+0000 7f7885d9c640 1 --2- 172.21.15.3:0/2702358823 >> v2:172.21.15.3:6801/2889521405 conn(0x1b1b310 0x1bae800 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:39.053 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.051+0000 7f7885d9c640 1 -- 172.21.15.3:0/2702358823 <== osd.0 v2:172.21.15.3:6801/2889521405 1 ==== osd_op_reply(1 obj7 [writefull 0~12875] v725'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f787800c430 con 0x1b1b310 2024-09-22T04:05:39.053 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.051+0000 7f788d452ec0 1 -- 172.21.15.3:0/2702358823 >> v2:172.21.15.3:6801/2889521405 conn(0x1b1b310 msgr2=0x1bae800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:39.054 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.051+0000 7f788d452ec0 1 --2- 172.21.15.3:0/2702358823 >> v2:172.21.15.3:6801/2889521405 conn(0x1b1b310 0x1bae800 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.054 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.051+0000 7f788d452ec0 1 -- 172.21.15.3:0/2702358823 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f786c03d360 msgr2=0x7f786c03f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:39.054 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.051+0000 7f788d452ec0 1 --2- 172.21.15.3:0/2702358823 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f786c03d360 0x7f786c03f870 crc :-1 s=READY pgs=3860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.054 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.051+0000 7f788d452ec0 1 -- 172.21.15.3:0/2702358823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b1a7f0 msgr2=0x1b07e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:39.054 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.051+0000 7f788d452ec0 1 --2- 172.21.15.3:0/2702358823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b1a7f0 0x1b07e10 crc :-1 s=READY pgs=12973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.054 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.052+0000 7f788d452ec0 1 -- 172.21.15.3:0/2702358823 shutdown_connections 2024-09-22T04:05:39.054 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.052+0000 7f788d452ec0 1 --2- 172.21.15.3:0/2702358823 >> v2:172.21.15.3:6801/2889521405 conn(0x1b1b310 0x1bae800 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.054 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.052+0000 7f788d452ec0 1 --2- 172.21.15.3:0/2702358823 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f786c03d360 0x7f786c03f870 unknown :-1 s=CLOSED pgs=3860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.054 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.052+0000 7f788d452ec0 1 --2- 172.21.15.3:0/2702358823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b1a7f0 0x1b07e10 unknown :-1 s=CLOSED pgs=12973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.054 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.052+0000 7f788d452ec0 1 -- 172.21.15.3:0/2702358823 >> 172.21.15.3:0/2702358823 conn(0x19ad770 msgr2=0x1a82650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:39.054 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.052+0000 7f788d452ec0 1 -- 172.21.15.3:0/2702358823 shutdown_connections 2024-09-22T04:05:39.054 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.052+0000 7f788d452ec0 1 -- 172.21.15.3:0/2702358823 wait complete. 2024-09-22T04:05:39.055 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:05:39.056 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p dd65dfbd-105b-4a15-8dc2-717b17ae7efb put obj8 /etc/passwd 2024-09-22T04:05:39.093 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.092+0000 7fef06d30ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:39.099 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.097+0000 7fef06d30ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:39.099 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.098+0000 7fef06d30ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:39.100 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.098+0000 7fef06d30ec0 1 Processor -- start 2024-09-22T04:05:39.100 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.099+0000 7fef06d30ec0 1 -- start start 2024-09-22T04:05:39.102 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.100+0000 7fef06d30ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xb487f0 0xbcf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:39.102 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.100+0000 7fef06d30ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0xa1b520 con 0xb487f0 2024-09-22T04:05:39.102 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.100+0000 7feeff03f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xb487f0 0xbcf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:39.102 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.100+0000 7feeff03f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xb487f0 0xbcf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42674/0 (socket says 172.21.15.3:42674) 2024-09-22T04:05:39.102 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.100+0000 7feeff03f640 1 -- 172.21.15.3:0/3623239513 learned_addr learned my addr 172.21.15.3:0/3623239513 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:39.102 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.100+0000 7feeff03f640 1 -- 172.21.15.3:0/3623239513 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x9b7e10 con 0xb487f0 2024-09-22T04:05:39.102 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.100+0000 7feeff03f640 1 --2- 172.21.15.3:0/3623239513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xb487f0 0xbcf730 crc :-1 s=READY pgs=12976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f30b2e98fdd248 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:39.102 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.100+0000 7feefd83c640 1 -- 172.21.15.3:0/3623239513 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7feee80061e0 con 0xb487f0 2024-09-22T04:05:39.102 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.101+0000 7feefd83c640 1 -- 172.21.15.3:0/3623239513 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7feee8006340 con 0xb487f0 2024-09-22T04:05:39.103 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.101+0000 7feefd83c640 1 -- 172.21.15.3:0/3623239513 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7feee8004880 con 0xb487f0 2024-09-22T04:05:39.104 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.101+0000 7fef06d30ec0 1 -- 172.21.15.3:0/3623239513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xb487f0 msgr2=0xbcf730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:39.104 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.101+0000 7fef06d30ec0 1 --2- 172.21.15.3:0/3623239513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xb487f0 0xbcf730 crc :-1 s=READY pgs=12976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.104 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.101+0000 7fef06d30ec0 1 -- 172.21.15.3:0/3623239513 shutdown_connections 2024-09-22T04:05:39.104 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.101+0000 7fef06d30ec0 1 --2- 172.21.15.3:0/3623239513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xb487f0 0xbcf730 unknown :-1 s=CLOSED pgs=12976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.104 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.101+0000 7fef06d30ec0 1 -- 172.21.15.3:0/3623239513 >> 172.21.15.3:0/3623239513 conn(0x9db770 msgr2=0xab0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:39.104 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.101+0000 7fef06d30ec0 1 -- 172.21.15.3:0/3623239513 shutdown_connections 2024-09-22T04:05:39.104 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.101+0000 7fef06d30ec0 1 -- 172.21.15.3:0/3623239513 wait complete. 2024-09-22T04:05:39.104 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.102+0000 7fef06d30ec0 1 Processor -- start 2024-09-22T04:05:39.104 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.102+0000 7fef06d30ec0 1 -- start start 2024-09-22T04:05:39.106 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.104+0000 7fef06d30ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xb487f0 0xbde950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:39.106 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.104+0000 7fef06d30ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0xa19280 con 0xb487f0 2024-09-22T04:05:39.106 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.104+0000 7feeff03f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xb487f0 0xbde950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:39.106 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.104+0000 7feeff03f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xb487f0 0xbde950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42680/0 (socket says 172.21.15.3:42680) 2024-09-22T04:05:39.106 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.104+0000 7feeff03f640 1 -- 172.21.15.3:0/1117668887 learned_addr learned my addr 172.21.15.3:0/1117668887 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:39.106 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.104+0000 7feeff03f640 1 -- 172.21.15.3:0/1117668887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x9a9920 con 0xb487f0 2024-09-22T04:05:39.106 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.104+0000 7feeff03f640 1 --2- 172.21.15.3:0/1117668887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xb487f0 0xbde950 crc :-1 s=READY pgs=12977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:39.106 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.104+0000 7feefc83a640 1 -- 172.21.15.3:0/1117668887 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7feee8001b70 con 0xb487f0 2024-09-22T04:05:39.106 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.104+0000 7feefc83a640 1 -- 172.21.15.3:0/1117668887 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7feee8001cd0 con 0xb487f0 2024-09-22T04:05:39.106 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.104+0000 7fef06d30ec0 1 -- 172.21.15.3:0/1117668887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x9a9380 con 0xb487f0 2024-09-22T04:05:39.106 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.104+0000 7feefc83a640 1 -- 172.21.15.3:0/1117668887 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7feee8001f70 con 0xb487f0 2024-09-22T04:05:39.106 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.105+0000 7fef06d30ec0 1 -- 172.21.15.3:0/1117668887 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0xbdc6a0 con 0xb487f0 2024-09-22T04:05:39.107 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.106+0000 7feefc83a640 1 -- 172.21.15.3:0/1117668887 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7feee8014410 con 0xb487f0 2024-09-22T04:05:39.108 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.106+0000 7feefc83a640 1 --2- 172.21.15.3:0/1117668887 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7feee403d360 0x7feee403f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:39.108 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.106+0000 7feefc83a640 1 -- 172.21.15.3:0/1117668887 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(725..725 src has 590..725) ==== 4207+0+0 (crc 0 0 0) 0x7feee804ff10 con 0xb487f0 2024-09-22T04:05:39.108 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.106+0000 7feefe83e640 1 --2- 172.21.15.3:0/1117668887 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7feee403d360 0x7feee403f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:39.108 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.106+0000 7fef06d30ec0 1 --2- 172.21.15.3:0/1117668887 >> v2:172.21.15.3:6801/2889521405 conn(0xb49310 0xb365e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:39.108 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.106+0000 7fef06d30ec0 1 -- 172.21.15.3:0/1117668887 --> v2:172.21.15.3:6801/2889521405 -- osd_op(unknown.0.0:1 200.5 200:a557efb1:::obj8:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e725) -- 0xb382b0 con 0xb49310 2024-09-22T04:05:39.108 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.107+0000 7feefe83e640 1 --2- 172.21.15.3:0/1117668887 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7feee403d360 0x7feee403f870 crc :-1 s=READY pgs=3861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:39.108 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.107+0000 7feeff840640 1 --2- 172.21.15.3:0/1117668887 >> v2:172.21.15.3:6801/2889521405 conn(0xb49310 0xb365e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:05:39.110 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.109+0000 7feeff840640 1 --2- 172.21.15.3:0/1117668887 >> v2:172.21.15.3:6801/2889521405 conn(0xb49310 0xb365e0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:39.133 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.131+0000 7feeff840640 1 -- 172.21.15.3:0/1117668887 <== osd.0 v2:172.21.15.3:6801/2889521405 1 ==== osd_op_reply(1 obj8 [writefull 0~12875] v725'1 uv1 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7feef000c430 con 0xb49310 2024-09-22T04:05:39.134 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.132+0000 7fef06d30ec0 1 -- 172.21.15.3:0/1117668887 >> v2:172.21.15.3:6801/2889521405 conn(0xb49310 msgr2=0xb365e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:39.134 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.132+0000 7fef06d30ec0 1 --2- 172.21.15.3:0/1117668887 >> v2:172.21.15.3:6801/2889521405 conn(0xb49310 0xb365e0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.134 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.132+0000 7fef06d30ec0 1 -- 172.21.15.3:0/1117668887 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7feee403d360 msgr2=0x7feee403f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:39.134 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.132+0000 7fef06d30ec0 1 --2- 172.21.15.3:0/1117668887 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7feee403d360 0x7feee403f870 crc :-1 s=READY pgs=3861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.134 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.132+0000 7fef06d30ec0 1 -- 172.21.15.3:0/1117668887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xb487f0 msgr2=0xbde950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:39.134 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.132+0000 7fef06d30ec0 1 --2- 172.21.15.3:0/1117668887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xb487f0 0xbde950 crc :-1 s=READY pgs=12977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.134 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.132+0000 7fef06d30ec0 1 -- 172.21.15.3:0/1117668887 shutdown_connections 2024-09-22T04:05:39.134 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.132+0000 7fef06d30ec0 1 --2- 172.21.15.3:0/1117668887 >> v2:172.21.15.3:6801/2889521405 conn(0xb49310 0xb365e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.134 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.132+0000 7fef06d30ec0 1 --2- 172.21.15.3:0/1117668887 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7feee403d360 0x7feee403f870 unknown :-1 s=CLOSED pgs=3861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.134 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.132+0000 7fef06d30ec0 1 --2- 172.21.15.3:0/1117668887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xb487f0 0xbde950 unknown :-1 s=CLOSED pgs=12977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.134 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.132+0000 7fef06d30ec0 1 -- 172.21.15.3:0/1117668887 >> 172.21.15.3:0/1117668887 conn(0x9db770 msgr2=0xab0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:39.134 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.132+0000 7fef06d30ec0 1 -- 172.21.15.3:0/1117668887 shutdown_connections 2024-09-22T04:05:39.135 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.132+0000 7fef06d30ec0 1 -- 172.21.15.3:0/1117668887 wait complete. 2024-09-22T04:05:39.135 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:05:39.135 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p dd65dfbd-105b-4a15-8dc2-717b17ae7efb put obj9 /etc/passwd 2024-09-22T04:05:39.173 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.172+0000 7f386bae7ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:39.179 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.178+0000 7f386bae7ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:39.179 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.178+0000 7f386bae7ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:39.180 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.179+0000 7f386bae7ec0 1 Processor -- start 2024-09-22T04:05:39.180 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.179+0000 7f386bae7ec0 1 -- start start 2024-09-22T04:05:39.182 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.180+0000 7f386bae7ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xdb58a0 0xdb3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:39.182 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.180+0000 7f386bae7ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0xc88520 con 0xdb58a0 2024-09-22T04:05:39.182 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.180+0000 7f385effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xdb58a0 0xdb3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:39.182 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.180+0000 7f385effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xdb58a0 0xdb3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42694/0 (socket says 172.21.15.3:42694) 2024-09-22T04:05:39.182 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.180+0000 7f385effd640 1 -- 172.21.15.3:0/369181949 learned_addr learned my addr 172.21.15.3:0/369181949 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:39.182 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.181+0000 7f385effd640 1 -- 172.21.15.3:0/369181949 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xc24e10 con 0xdb58a0 2024-09-22T04:05:39.182 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.181+0000 7f385effd640 1 --2- 172.21.15.3:0/369181949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xdb58a0 0xdb3e50 crc :-1 s=READY pgs=12980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aac19ff070dc031 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:39.182 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.181+0000 7f385d7fa640 1 -- 172.21.15.3:0/369181949 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f385002dcc0 con 0xdb58a0 2024-09-22T04:05:39.183 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.181+0000 7f385d7fa640 1 -- 172.21.15.3:0/369181949 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f3850037370 con 0xdb58a0 2024-09-22T04:05:39.183 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.181+0000 7f385d7fa640 1 -- 172.21.15.3:0/369181949 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3850037610 con 0xdb58a0 2024-09-22T04:05:39.184 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.182+0000 7f386bae7ec0 1 -- 172.21.15.3:0/369181949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xdb58a0 msgr2=0xdb3e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:39.184 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.182+0000 7f386bae7ec0 1 --2- 172.21.15.3:0/369181949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xdb58a0 0xdb3e50 crc :-1 s=READY pgs=12980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.184 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.182+0000 7f386bae7ec0 1 -- 172.21.15.3:0/369181949 shutdown_connections 2024-09-22T04:05:39.184 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.182+0000 7f386bae7ec0 1 --2- 172.21.15.3:0/369181949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xdb58a0 0xdb3e50 unknown :-1 s=CLOSED pgs=12980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.184 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.182+0000 7f386bae7ec0 1 -- 172.21.15.3:0/369181949 >> 172.21.15.3:0/369181949 conn(0xc48770 msgr2=0xd24930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:39.184 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.182+0000 7f386bae7ec0 1 -- 172.21.15.3:0/369181949 shutdown_connections 2024-09-22T04:05:39.184 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.182+0000 7f386bae7ec0 1 -- 172.21.15.3:0/369181949 wait complete. 2024-09-22T04:05:39.184 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.183+0000 7f386bae7ec0 1 Processor -- start 2024-09-22T04:05:39.184 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.183+0000 7f386bae7ec0 1 -- start start 2024-09-22T04:05:39.186 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.184+0000 7f386bae7ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xdb58a0 0xd9fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:39.186 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.184+0000 7f386bae7ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0xc86280 con 0xdb58a0 2024-09-22T04:05:39.186 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.184+0000 7f385effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xdb58a0 0xd9fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:39.186 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.184+0000 7f385effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xdb58a0 0xd9fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42698/0 (socket says 172.21.15.3:42698) 2024-09-22T04:05:39.186 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.184+0000 7f385effd640 1 -- 172.21.15.3:0/1596649757 learned_addr learned my addr 172.21.15.3:0/1596649757 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:39.186 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.184+0000 7f385effd640 1 -- 172.21.15.3:0/1596649757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0xc16920 con 0xdb58a0 2024-09-22T04:05:39.186 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.185+0000 7f385effd640 1 --2- 172.21.15.3:0/1596649757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xdb58a0 0xd9fa70 crc :-1 s=READY pgs=12981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:39.186 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.185+0000 7f3847fff640 1 -- 172.21.15.3:0/1596649757 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f385003e040 con 0xdb58a0 2024-09-22T04:05:39.187 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.185+0000 7f3847fff640 1 -- 172.21.15.3:0/1596649757 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f385002db90 con 0xdb58a0 2024-09-22T04:05:39.187 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.185+0000 7f386bae7ec0 1 -- 172.21.15.3:0/1596649757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0xc16380 con 0xdb58a0 2024-09-22T04:05:39.187 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.185+0000 7f3847fff640 1 -- 172.21.15.3:0/1596649757 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f3850037d40 con 0xdb58a0 2024-09-22T04:05:39.187 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.185+0000 7f386bae7ec0 1 -- 172.21.15.3:0/1596649757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0xda2310 con 0xdb58a0 2024-09-22T04:05:39.187 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.186+0000 7f3847fff640 1 -- 172.21.15.3:0/1596649757 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f3850041040 con 0xdb58a0 2024-09-22T04:05:39.188 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.186+0000 7f3847fff640 1 --2- 172.21.15.3:0/1596649757 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f384c03d3b0 0x7f384c03f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:39.188 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.186+0000 7f3847fff640 1 -- 172.21.15.3:0/1596649757 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(725..725 src has 590..725) ==== 4207+0+0 (crc 0 0 0) 0x7f3850032040 con 0xdb58a0 2024-09-22T04:05:39.188 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.186+0000 7f386bae7ec0 1 --2- 172.21.15.3:0/1596649757 >> v2:172.21.15.123:6806/4206675901 conn(0xdb61f0 0xe4a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:39.188 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.186+0000 7f385e7fc640 1 --2- 172.21.15.3:0/1596649757 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f384c03d3b0 0x7f384c03f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:39.188 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.186+0000 7f386bae7ec0 1 -- 172.21.15.3:0/1596649757 --> v2:172.21.15.123:6806/4206675901 -- osd_op(unknown.0.0:1 200.7 200:fe351e27:::obj9:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e725) -- 0xda29a0 con 0xdb61f0 2024-09-22T04:05:39.188 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.187+0000 7f385e7fc640 1 --2- 172.21.15.3:0/1596649757 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f384c03d3b0 0x7f384c03f8c0 crc :-1 s=READY pgs=3862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:39.188 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.187+0000 7f385f7fe640 1 --2- 172.21.15.3:0/1596649757 >> v2:172.21.15.123:6806/4206675901 conn(0xdb61f0 0xe4a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:05:39.192 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.191+0000 7f385f7fe640 1 --2- 172.21.15.3:0/1596649757 >> v2:172.21.15.123:6806/4206675901 conn(0xdb61f0 0xe4a6f0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:39.214 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.212+0000 7f385f7fe640 1 -- 172.21.15.3:0/1596649757 <== osd.3 v2:172.21.15.123:6806/4206675901 1 ==== osd_op_reply(1 obj9 [writefull 0~12875] v725'2 uv2 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f385800a430 con 0xdb61f0 2024-09-22T04:05:39.214 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.213+0000 7f386bae7ec0 1 -- 172.21.15.3:0/1596649757 >> v2:172.21.15.123:6806/4206675901 conn(0xdb61f0 msgr2=0xe4a6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:39.215 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.213+0000 7f386bae7ec0 1 --2- 172.21.15.3:0/1596649757 >> v2:172.21.15.123:6806/4206675901 conn(0xdb61f0 0xe4a6f0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.215 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.213+0000 7f386bae7ec0 1 -- 172.21.15.3:0/1596649757 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f384c03d3b0 msgr2=0x7f384c03f8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:39.215 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.213+0000 7f386bae7ec0 1 --2- 172.21.15.3:0/1596649757 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f384c03d3b0 0x7f384c03f8c0 crc :-1 s=READY pgs=3862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.215 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.213+0000 7f386bae7ec0 1 -- 172.21.15.3:0/1596649757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xdb58a0 msgr2=0xd9fa70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:39.215 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.213+0000 7f386bae7ec0 1 --2- 172.21.15.3:0/1596649757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xdb58a0 0xd9fa70 crc :-1 s=READY pgs=12981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.215 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.213+0000 7f386bae7ec0 1 -- 172.21.15.3:0/1596649757 shutdown_connections 2024-09-22T04:05:39.215 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.213+0000 7f386bae7ec0 1 --2- 172.21.15.3:0/1596649757 >> v2:172.21.15.123:6806/4206675901 conn(0xdb61f0 0xe4a6f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.215 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.213+0000 7f386bae7ec0 1 --2- 172.21.15.3:0/1596649757 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f384c03d3b0 0x7f384c03f8c0 unknown :-1 s=CLOSED pgs=3862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.215 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.213+0000 7f386bae7ec0 1 --2- 172.21.15.3:0/1596649757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0xdb58a0 0xd9fa70 unknown :-1 s=CLOSED pgs=12981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.215 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.213+0000 7f386bae7ec0 1 -- 172.21.15.3:0/1596649757 >> 172.21.15.3:0/1596649757 conn(0xc48770 msgr2=0xd234f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:39.215 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.213+0000 7f386bae7ec0 1 -- 172.21.15.3:0/1596649757 shutdown_connections 2024-09-22T04:05:39.215 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.213+0000 7f386bae7ec0 1 -- 172.21.15.3:0/1596649757 wait complete. 2024-09-22T04:05:39.216 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in `seq 1 10` 2024-09-22T04:05:39.217 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p dd65dfbd-105b-4a15-8dc2-717b17ae7efb put obj10 /etc/passwd 2024-09-22T04:05:39.255 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.254+0000 7fcf170fdec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:39.260 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.259+0000 7fcf170fdec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:39.260 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.259+0000 7fcf170fdec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:05:39.261 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.260+0000 7fcf170fdec0 1 Processor -- start 2024-09-22T04:05:39.261 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.260+0000 7fcf170fdec0 1 -- start start 2024-09-22T04:05:39.263 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.262+0000 7fcf170fdec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x23f18a0 0x2478610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:39.263 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.262+0000 7fcf170fdec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x22c4520 con 0x23f18a0 2024-09-22T04:05:39.264 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.262+0000 7fcf0f43f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x23f18a0 0x2478610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:39.264 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.262+0000 7fcf0f43f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x23f18a0 0x2478610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42714/0 (socket says 172.21.15.3:42714) 2024-09-22T04:05:39.264 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.262+0000 7fcf0f43f640 1 -- 172.21.15.3:0/2504944116 learned_addr learned my addr 172.21.15.3:0/2504944116 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:39.264 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.262+0000 7fcf0f43f640 1 -- 172.21.15.3:0/2504944116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x2260e10 con 0x23f18a0 2024-09-22T04:05:39.264 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.262+0000 7fcf0f43f640 1 --2- 172.21.15.3:0/2504944116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x23f18a0 0x2478610 crc :-1 s=READY pgs=12984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49c8b8a50aafbfd server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:39.264 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.262+0000 7fcf0dc3c640 1 -- 172.21.15.3:0/2504944116 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcef802dcc0 con 0x23f18a0 2024-09-22T04:05:39.264 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.262+0000 7fcf0dc3c640 1 -- 172.21.15.3:0/2504944116 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fcef8037370 con 0x23f18a0 2024-09-22T04:05:39.264 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.262+0000 7fcf0dc3c640 1 -- 172.21.15.3:0/2504944116 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcef8037610 con 0x23f18a0 2024-09-22T04:05:39.265 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.263+0000 7fcf170fdec0 1 -- 172.21.15.3:0/2504944116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x23f18a0 msgr2=0x2478610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:39.265 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.263+0000 7fcf170fdec0 1 --2- 172.21.15.3:0/2504944116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x23f18a0 0x2478610 crc :-1 s=READY pgs=12984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.265 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.263+0000 7fcf170fdec0 1 -- 172.21.15.3:0/2504944116 shutdown_connections 2024-09-22T04:05:39.265 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.263+0000 7fcf170fdec0 1 --2- 172.21.15.3:0/2504944116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x23f18a0 0x2478610 unknown :-1 s=CLOSED pgs=12984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.266 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.263+0000 7fcf170fdec0 1 -- 172.21.15.3:0/2504944116 >> 172.21.15.3:0/2504944116 conn(0x2284770 msgr2=0x23efe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:39.266 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.263+0000 7fcf170fdec0 1 -- 172.21.15.3:0/2504944116 shutdown_connections 2024-09-22T04:05:39.266 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.263+0000 7fcf170fdec0 1 -- 172.21.15.3:0/2504944116 wait complete. 2024-09-22T04:05:39.266 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.264+0000 7fcf170fdec0 1 Processor -- start 2024-09-22T04:05:39.266 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.264+0000 7fcf170fdec0 1 -- start start 2024-09-22T04:05:39.267 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.266+0000 7fcf170fdec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x23f18a0 0x2487700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:39.267 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.266+0000 7fcf170fdec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x22c2280 con 0x23f18a0 2024-09-22T04:05:39.268 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.266+0000 7fcf0f43f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x23f18a0 0x2487700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:39.268 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.266+0000 7fcf0f43f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x23f18a0 0x2487700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42718/0 (socket says 172.21.15.3:42718) 2024-09-22T04:05:39.268 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.266+0000 7fcf0f43f640 1 -- 172.21.15.3:0/3256987218 learned_addr learned my addr 172.21.15.3:0/3256987218 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:05:39.268 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.266+0000 7fcf0f43f640 1 -- 172.21.15.3:0/3256987218 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x2252920 con 0x23f18a0 2024-09-22T04:05:39.268 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.266+0000 7fcf0f43f640 1 --2- 172.21.15.3:0/3256987218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x23f18a0 0x2487700 crc :-1 s=READY pgs=12985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:39.268 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.266+0000 7fcf0cc3a640 1 -- 172.21.15.3:0/3256987218 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcef802db90 con 0x23f18a0 2024-09-22T04:05:39.268 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.266+0000 7fcf0cc3a640 1 -- 172.21.15.3:0/3256987218 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fcef8037c90 con 0x23f18a0 2024-09-22T04:05:39.268 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.266+0000 7fcf170fdec0 1 -- 172.21.15.3:0/3256987218 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x2252380 con 0x23f18a0 2024-09-22T04:05:39.268 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.266+0000 7fcf0cc3a640 1 -- 172.21.15.3:0/3256987218 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fcef8037e00 con 0x23f18a0 2024-09-22T04:05:39.268 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.266+0000 7fcf170fdec0 1 -- 172.21.15.3:0/3256987218 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x2357550 con 0x23f18a0 2024-09-22T04:05:39.269 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.267+0000 7fcf0cc3a640 1 -- 172.21.15.3:0/3256987218 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fcef8039450 con 0x23f18a0 2024-09-22T04:05:39.269 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.268+0000 7fcf0cc3a640 1 --2- 172.21.15.3:0/3256987218 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fcef403d360 0x7fcef403f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:39.269 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.268+0000 7fcf0cc3a640 1 -- 172.21.15.3:0/3256987218 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(725..725 src has 590..725) ==== 4207+0+0 (crc 0 0 0) 0x7fcef8032040 con 0x23f18a0 2024-09-22T04:05:39.269 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.268+0000 7fcf170fdec0 1 --2- 172.21.15.3:0/3256987218 >> v2:172.21.15.3:6801/2889521405 conn(0x23f21f0 0x235d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:05:39.269 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.268+0000 7fcf170fdec0 1 -- 172.21.15.3:0/3256987218 --> v2:172.21.15.3:6801/2889521405 -- osd_op(unknown.0.0:1 200.b 200:d1337354:::obj10:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e725) -- 0x23dbba0 con 0x23f21f0 2024-09-22T04:05:39.269 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.268+0000 7fcf0ec3e640 1 --2- 172.21.15.3:0/3256987218 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fcef403d360 0x7fcef403f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:05:39.270 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.269+0000 7fcf0ec3e640 1 --2- 172.21.15.3:0/3256987218 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fcef403d360 0x7fcef403f870 crc :-1 s=READY pgs=3863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:39.270 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.269+0000 7fcf0fc40640 1 --2- 172.21.15.3:0/3256987218 >> v2:172.21.15.3:6801/2889521405 conn(0x23f21f0 0x235d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:05:39.273 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.272+0000 7fcf0fc40640 1 --2- 172.21.15.3:0/3256987218 >> v2:172.21.15.3:6801/2889521405 conn(0x23f21f0 0x235d1d0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:05:39.300 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.298+0000 7fcf0fc40640 1 -- 172.21.15.3:0/3256987218 <== osd.0 v2:172.21.15.3:6801/2889521405 1 ==== osd_op_reply(1 obj10 [writefull 0~12875] v725'1 uv1 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fcf0000a430 con 0x23f21f0 2024-09-22T04:05:39.300 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.298+0000 7fcf170fdec0 1 -- 172.21.15.3:0/3256987218 >> v2:172.21.15.3:6801/2889521405 conn(0x23f21f0 msgr2=0x235d1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:39.300 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.298+0000 7fcf170fdec0 1 --2- 172.21.15.3:0/3256987218 >> v2:172.21.15.3:6801/2889521405 conn(0x23f21f0 0x235d1d0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.300 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.298+0000 7fcf170fdec0 1 -- 172.21.15.3:0/3256987218 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fcef403d360 msgr2=0x7fcef403f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:39.300 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.298+0000 7fcf170fdec0 1 --2- 172.21.15.3:0/3256987218 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fcef403d360 0x7fcef403f870 crc :-1 s=READY pgs=3863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.300 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.298+0000 7fcf170fdec0 1 -- 172.21.15.3:0/3256987218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x23f18a0 msgr2=0x2487700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:05:39.300 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.298+0000 7fcf170fdec0 1 --2- 172.21.15.3:0/3256987218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x23f18a0 0x2487700 crc :-1 s=READY pgs=12985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.300 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.298+0000 7fcf170fdec0 1 -- 172.21.15.3:0/3256987218 shutdown_connections 2024-09-22T04:05:39.300 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.298+0000 7fcf170fdec0 1 --2- 172.21.15.3:0/3256987218 >> v2:172.21.15.3:6801/2889521405 conn(0x23f21f0 0x235d1d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.301 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.298+0000 7fcf170fdec0 1 --2- 172.21.15.3:0/3256987218 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fcef403d360 0x7fcef403f870 unknown :-1 s=CLOSED pgs=3863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.301 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.298+0000 7fcf170fdec0 1 --2- 172.21.15.3:0/3256987218 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x23f18a0 0x2487700 unknown :-1 s=CLOSED pgs=12985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:05:39.301 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.298+0000 7fcf170fdec0 1 -- 172.21.15.3:0/3256987218 >> 172.21.15.3:0/3256987218 conn(0x2284770 msgr2=0x23ef490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:05:39.301 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.298+0000 7fcf170fdec0 1 -- 172.21.15.3:0/3256987218 shutdown_connections 2024-09-22T04:05:39.301 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:05:39.298+0000 7fcf170fdec0 1 -- 172.21.15.3:0/3256987218 wait complete. 2024-09-22T04:05:39.302 INFO:tasks.workunit.client.0.smithi003.stderr:+ sleep 30 2024-09-22T04:06:09.303 INFO:tasks.workunit.client.0.smithi003.stderr:+ rados -p 7c1c124a-aa8e-411d-9731-1f1389755458 put threemore /etc/passwd 2024-09-22T04:06:09.349 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.347+0000 7ff039a43ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:09.355 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.354+0000 7ff039a43ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:09.355 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.354+0000 7ff039a43ec0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:09.356 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.355+0000 7ff039a43ec0 1 Processor -- start 2024-09-22T04:06:09.356 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.355+0000 7ff039a43ec0 1 -- start start 2024-09-22T04:06:09.358 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.356+0000 7ff039a43ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b44f80 0x1bc26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:09.358 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.356+0000 7ff039a43ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x1aa9520 con 0x1b44f80 2024-09-22T04:06:09.358 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.356+0000 7ff031b9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b44f80 0x1bc26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:09.358 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.356+0000 7ff031b9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b44f80 0x1bc26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36402/0 (socket says 172.21.15.3:36402) 2024-09-22T04:06:09.358 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.356+0000 7ff031b9b640 1 -- 172.21.15.3:0/2198844981 learned_addr learned my addr 172.21.15.3:0/2198844981 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:09.358 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.357+0000 7ff031b9b640 1 -- 172.21.15.3:0/2198844981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1a45e10 con 0x1b44f80 2024-09-22T04:06:09.358 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.357+0000 7ff031b9b640 1 --2- 172.21.15.3:0/2198844981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b44f80 0x1bc26a0 crc :-1 s=READY pgs=12988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8dba290f3b08f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:09.359 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.357+0000 7ff02bfff640 1 -- 172.21.15.3:0/2198844981 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff01c02d9a0 con 0x1b44f80 2024-09-22T04:06:09.359 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.357+0000 7ff02bfff640 1 -- 172.21.15.3:0/2198844981 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff01c02db80 con 0x1b44f80 2024-09-22T04:06:09.360 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.357+0000 7ff02bfff640 1 -- 172.21.15.3:0/2198844981 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff01c02de20 con 0x1b44f80 2024-09-22T04:06:09.360 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.358+0000 7ff039a43ec0 1 -- 172.21.15.3:0/2198844981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b44f80 msgr2=0x1bc26a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:09.360 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.358+0000 7ff039a43ec0 1 --2- 172.21.15.3:0/2198844981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b44f80 0x1bc26a0 crc :-1 s=READY pgs=12988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:09.360 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.358+0000 7ff039a43ec0 1 -- 172.21.15.3:0/2198844981 shutdown_connections 2024-09-22T04:06:09.360 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.358+0000 7ff039a43ec0 1 --2- 172.21.15.3:0/2198844981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b44f80 0x1bc26a0 unknown :-1 s=CLOSED pgs=12988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:09.360 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.358+0000 7ff039a43ec0 1 -- 172.21.15.3:0/2198844981 >> 172.21.15.3:0/2198844981 conn(0x1a69770 msgr2=0x1b41df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:09.360 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.358+0000 7ff039a43ec0 1 -- 172.21.15.3:0/2198844981 shutdown_connections 2024-09-22T04:06:09.360 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.358+0000 7ff039a43ec0 1 -- 172.21.15.3:0/2198844981 wait complete. 2024-09-22T04:06:09.361 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.359+0000 7ff039a43ec0 1 Processor -- start 2024-09-22T04:06:09.361 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.359+0000 7ff039a43ec0 1 -- start start 2024-09-22T04:06:09.362 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.360+0000 7ff039a43ec0 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b44f80 0x1bca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:09.363 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.360+0000 7ff039a43ec0 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x1aa7280 con 0x1b44f80 2024-09-22T04:06:09.363 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.361+0000 7ff031b9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b44f80 0x1bca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:09.363 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.361+0000 7ff031b9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b44f80 0x1bca370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36406/0 (socket says 172.21.15.3:36406) 2024-09-22T04:06:09.363 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.361+0000 7ff031b9b640 1 -- 172.21.15.3:0/2273370756 learned_addr learned my addr 172.21.15.3:0/2273370756 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:09.363 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.361+0000 7ff031b9b640 1 -- 172.21.15.3:0/2273370756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x1a37920 con 0x1b44f80 2024-09-22T04:06:09.363 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.361+0000 7ff031b9b640 1 --2- 172.21.15.3:0/2273370756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b44f80 0x1bca370 crc :-1 s=READY pgs=12989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:09.363 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.361+0000 7ff02affd640 1 -- 172.21.15.3:0/2273370756 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff01c02dcf0 con 0x1b44f80 2024-09-22T04:06:09.363 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.361+0000 7ff02affd640 1 -- 172.21.15.3:0/2273370756 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7ff01c035a10 con 0x1b44f80 2024-09-22T04:06:09.363 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.361+0000 7ff039a43ec0 1 -- 172.21.15.3:0/2273370756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x1a37380 con 0x1b44f80 2024-09-22T04:06:09.363 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.361+0000 7ff02affd640 1 -- 172.21.15.3:0/2273370756 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7ff01c035b80 con 0x1b44f80 2024-09-22T04:06:09.363 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.361+0000 7ff039a43ec0 1 -- 172.21.15.3:0/2273370756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x1bc85a0 con 0x1b44f80 2024-09-22T04:06:09.364 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.362+0000 7ff02affd640 1 -- 172.21.15.3:0/2273370756 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7ff01c035d30 con 0x1b44f80 2024-09-22T04:06:09.364 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.363+0000 7ff02affd640 1 --2- 172.21.15.3:0/2273370756 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7ff01803d360 0x7ff01803f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:09.365 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.363+0000 7ff02affd640 1 -- 172.21.15.3:0/2273370756 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(727..727 src has 590..727) ==== 4207+0+0 (crc 0 0 0) 0x7ff01c032040 con 0x1b44f80 2024-09-22T04:06:09.365 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.363+0000 7ff02affd640 1 -- 172.21.15.3:0/2273370756 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=728}) -- 0x7ff018043df0 con 0x1b44f80 2024-09-22T04:06:09.365 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.363+0000 7ff039a43ec0 1 --2- 172.21.15.3:0/2273370756 >> v2:172.21.15.123:6813/891966684 conn(0x1b45cd0 0x1bc76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:09.365 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.363+0000 7ff039a43ec0 1 -- 172.21.15.3:0/2273370756 --> v2:172.21.15.123:6813/891966684 -- osd_op(unknown.0.0:1 199.2 199:47cad802:::threemore:head [writefull 0~12875 in=12875b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e727) -- 0x1bd72a0 con 0x1b45cd0 2024-09-22T04:06:09.365 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.363+0000 7ff03139a640 1 --2- 172.21.15.3:0/2273370756 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7ff01803d360 0x7ff01803f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:09.365 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.364+0000 7ff03139a640 1 --2- 172.21.15.3:0/2273370756 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7ff01803d360 0x7ff01803f870 crc :-1 s=READY pgs=3864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:09.365 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.364+0000 7ff03239c640 1 --2- 172.21.15.3:0/2273370756 >> v2:172.21.15.123:6813/891966684 conn(0x1b45cd0 0x1bc76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:06:09.370 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.369+0000 7ff03239c640 1 --2- 172.21.15.3:0/2273370756 >> v2:172.21.15.123:6813/891966684 conn(0x1b45cd0 0x1bc76e0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:09.402 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.400+0000 7ff03239c640 1 -- 172.21.15.3:0/2273370756 <== osd.2 v2:172.21.15.123:6813/891966684 1 ==== osd_op_reply(1 threemore [writefull 0~12875] v727'1 uv1 ondisk = 0) ==== 153+0+0 (crc 0 0 0) 0x7ff02400c430 con 0x1b45cd0 2024-09-22T04:06:09.402 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.400+0000 7ff039a43ec0 1 -- 172.21.15.3:0/2273370756 >> v2:172.21.15.123:6813/891966684 conn(0x1b45cd0 msgr2=0x1bc76e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:09.402 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.400+0000 7ff039a43ec0 1 --2- 172.21.15.3:0/2273370756 >> v2:172.21.15.123:6813/891966684 conn(0x1b45cd0 0x1bc76e0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:09.402 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.400+0000 7ff039a43ec0 1 -- 172.21.15.3:0/2273370756 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7ff01803d360 msgr2=0x7ff01803f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:09.402 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.400+0000 7ff039a43ec0 1 --2- 172.21.15.3:0/2273370756 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7ff01803d360 0x7ff01803f870 crc :-1 s=READY pgs=3864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:09.402 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.400+0000 7ff039a43ec0 1 -- 172.21.15.3:0/2273370756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b44f80 msgr2=0x1bca370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:09.402 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.400+0000 7ff039a43ec0 1 --2- 172.21.15.3:0/2273370756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b44f80 0x1bca370 crc :-1 s=READY pgs=12989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:09.402 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.400+0000 7ff039a43ec0 1 -- 172.21.15.3:0/2273370756 shutdown_connections 2024-09-22T04:06:09.402 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.400+0000 7ff039a43ec0 1 --2- 172.21.15.3:0/2273370756 >> v2:172.21.15.123:6813/891966684 conn(0x1b45cd0 0x1bc76e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:09.402 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.400+0000 7ff039a43ec0 1 --2- 172.21.15.3:0/2273370756 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7ff01803d360 0x7ff01803f870 unknown :-1 s=CLOSED pgs=3864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:09.402 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.400+0000 7ff039a43ec0 1 --2- 172.21.15.3:0/2273370756 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x1b44f80 0x1bca370 unknown :-1 s=CLOSED pgs=12989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:09.403 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.400+0000 7ff039a43ec0 1 -- 172.21.15.3:0/2273370756 >> 172.21.15.3:0/2273370756 conn(0x1a69770 msgr2=0x1bc26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:09.403 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.400+0000 7ff039a43ec0 1 -- 172.21.15.3:0/2273370756 shutdown_connections 2024-09-22T04:06:09.403 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.400+0000 7ff039a43ec0 1 -- 172.21.15.3:0/2273370756 wait complete. 2024-09-22T04:06:09.404 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool set-quota 7c1c124a-aa8e-411d-9731-1f1389755458 max_bytes 0 2024-09-22T04:06:09.506 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.496+0000 7fe0c34d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:09.507 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.506+0000 7fe0c34d1640 1 Processor -- start 2024-09-22T04:06:09.507 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.506+0000 7fe0c34d1640 1 -- start start 2024-09-22T04:06:09.510 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.508+0000 7fe0c34d1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe0bc133d80 0x7fe0bc134190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:09.510 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.508+0000 7fe0c34d1640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0bc0748f0 con 0x7fe0bc09da20 2024-09-22T04:06:09.510 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.508+0000 7fe0c34d1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe0bc0a1110 con 0x7fe0bc133d80 2024-09-22T04:06:09.510 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.508+0000 7fe0c126d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe0bc133d80 0x7fe0bc134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:09.510 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.508+0000 7fe0c126d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe0bc133d80 0x7fe0bc134190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36408/0 (socket says 172.21.15.3:36408) 2024-09-22T04:06:09.510 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.508+0000 7fe0c0a6c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe0bc09da20 0x7fe0bc09de30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49328/0 (socket says 172.21.15.3:49328) 2024-09-22T04:06:09.510 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.508+0000 7fe0c126d640 1 -- 172.21.15.3:0/1503826659 learned_addr learned my addr 172.21.15.3:0/1503826659 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:09.510 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.509+0000 7fe0c126d640 1 -- 172.21.15.3:0/1503826659 >> v1:172.21.15.3:6789/0 conn(0x7fe0bc09da20 legacy=0x7fe0bc09de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:09.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.509+0000 7fe0b3fff640 1 -- 172.21.15.3:0/1503826659 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2980179607 0 0) 0x7fe0bc0748f0 con 0x7fe0bc09da20 2024-09-22T04:06:09.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.509+0000 7fe0c126d640 1 -- 172.21.15.3:0/1503826659 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc09f5b0 con 0x7fe0bc133d80 2024-09-22T04:06:09.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.509+0000 7fe0c126d640 1 --2- 172.21.15.3:0/1503826659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc133d80 0x7fe0bc134190 crc :-1 s=READY pgs=12991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e968a69e059a843 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:09.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.509+0000 7fe0b3fff640 1 -- 172.21.15.3:0/1503826659 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe0b8009e40 con 0x7fe0bc133d80 2024-09-22T04:06:09.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.509+0000 7fe0b3fff640 1 -- 172.21.15.3:0/1503826659 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe0b800e230 con 0x7fe0bc133d80 2024-09-22T04:06:09.511 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.509+0000 7fe0b3fff640 1 -- 172.21.15.3:0/1503826659 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe0b800e4d0 con 0x7fe0bc133d80 2024-09-22T04:06:09.512 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.510+0000 7fe0c34d1640 1 -- 172.21.15.3:0/1503826659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc133d80 msgr2=0x7fe0bc134190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:09.512 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.510+0000 7fe0c34d1640 1 --2- 172.21.15.3:0/1503826659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc133d80 0x7fe0bc134190 crc :-1 s=READY pgs=12991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:09.512 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.510+0000 7fe0c34d1640 1 -- 172.21.15.3:0/1503826659 shutdown_connections 2024-09-22T04:06:09.512 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.510+0000 7fe0c34d1640 1 --2- 172.21.15.3:0/1503826659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc133d80 0x7fe0bc134190 unknown :-1 s=CLOSED pgs=12991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:09.512 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.510+0000 7fe0c34d1640 1 -- 172.21.15.3:0/1503826659 >> 172.21.15.3:0/1503826659 conn(0x7fe0bc0a6d10 msgr2=0x7fe0bc073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:09.512 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.510+0000 7fe0c34d1640 1 -- 172.21.15.3:0/1503826659 shutdown_connections 2024-09-22T04:06:09.512 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.510+0000 7fe0c34d1640 1 -- 172.21.15.3:0/1503826659 wait complete. 2024-09-22T04:06:09.512 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.510+0000 7fe0c34d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:09.513 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.512+0000 7fe0c34d1640 1 Processor -- start 2024-09-22T04:06:09.513 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.512+0000 7fe0c34d1640 1 -- start start 2024-09-22T04:06:09.515 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.513+0000 7fe0c34d1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc09da20 0x7fe0bc1208a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:09.515 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.513+0000 7fe0c34d1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc1338c0 con 0x7fe0bc09da20 2024-09-22T04:06:09.515 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.513+0000 7fe0c126d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc09da20 0x7fe0bc1208a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:09.515 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.513+0000 7fe0c126d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc09da20 0x7fe0bc1208a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36422/0 (socket says 172.21.15.3:36422) 2024-09-22T04:06:09.515 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.513+0000 7fe0c126d640 1 -- 172.21.15.3:0/1381094286 learned_addr learned my addr 172.21.15.3:0/1381094286 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:09.515 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.514+0000 7fe0c126d640 1 -- 172.21.15.3:0/1381094286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc123ed0 con 0x7fe0bc09da20 2024-09-22T04:06:09.516 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.514+0000 7fe0c126d640 1 --2- 172.21.15.3:0/1381094286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc09da20 0x7fe0bc1208a0 crc :-1 s=READY pgs=12992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:09.516 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.514+0000 7fe0b1ffb640 1 -- 172.21.15.3:0/1381094286 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe0b800ee10 con 0x7fe0bc09da20 2024-09-22T04:06:09.516 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.514+0000 7fe0b1ffb640 1 -- 172.21.15.3:0/1381094286 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe0b8015420 con 0x7fe0bc09da20 2024-09-22T04:06:09.516 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.514+0000 7fe0c34d1640 1 -- 172.21.15.3:0/1381094286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0bc122140 con 0x7fe0bc09da20 2024-09-22T04:06:09.516 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.514+0000 7fe0b1ffb640 1 -- 172.21.15.3:0/1381094286 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe0b80156c0 con 0x7fe0bc09da20 2024-09-22T04:06:09.516 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.514+0000 7fe0c34d1640 1 -- 172.21.15.3:0/1381094286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0bc1245f0 con 0x7fe0bc09da20 2024-09-22T04:06:09.517 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.515+0000 7fe0c34d1640 1 -- 172.21.15.3:0/1381094286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0bc0a5cc0 con 0x7fe0bc09da20 2024-09-22T04:06:09.517 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.515+0000 7fe0b1ffb640 1 -- 172.21.15.3:0/1381094286 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fe0b8015870 con 0x7fe0bc09da20 2024-09-22T04:06:09.517 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.516+0000 7fe0b1ffb640 1 --2- 172.21.15.3:0/1381094286 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe0a003ead0 0x7fe0a0040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:09.517 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.516+0000 7fe0b1ffb640 1 -- 172.21.15.3:0/1381094286 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(727..727 src has 590..727) ==== 4207+0+0 (crc 0 0 0) 0x7fe0b8051470 con 0x7fe0bc09da20 2024-09-22T04:06:09.517 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.516+0000 7fe0b1ffb640 1 -- 172.21.15.3:0/1381094286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=728}) -- 0x7fe0a0045560 con 0x7fe0bc09da20 2024-09-22T04:06:09.517 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.516+0000 7fe0c0a6c640 1 --2- 172.21.15.3:0/1381094286 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe0a003ead0 0x7fe0a0040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:09.518 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.516+0000 7fe0c0a6c640 1 --2- 172.21.15.3:0/1381094286 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe0a003ead0 0x7fe0a0040fe0 crc :-1 s=READY pgs=3865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:09.522 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.521+0000 7fe0b1ffb640 1 -- 172.21.15.3:0/1381094286 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7fe0b8014c60 con 0x7fe0bc09da20 2024-09-22T04:06:09.738 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.737+0000 7fe0c34d1640 1 -- 172.21.15.3:0/1381094286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_bytes", "val": "0"} v 0) -- 0x7fe0bc050fc0 con 0x7fe0bc09da20 2024-09-22T04:06:09.892 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.891+0000 7fe0b1ffb640 1 -- 172.21.15.3:0/1381094286 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 v728) ==== 217+0+0 (crc 0 0 0) 0x7fe0b80059f0 con 0x7fe0bc09da20 2024-09-22T04:06:09.895 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.894+0000 7fe0b1ffb640 1 -- 172.21.15.3:0/1381094286 <== mon.0 v2:172.21.15.3:3300/0 8 ==== osd_map(728..728 src has 590..728) ==== 628+0+0 (crc 0 0 0) 0x7fe0b8011e00 con 0x7fe0bc09da20 2024-09-22T04:06:09.895 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:09.894+0000 7fe0b1ffb640 1 -- 172.21.15.3:0/1381094286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=729}) -- 0x7fe0a0046550 con 0x7fe0bc09da20 2024-09-22T04:06:10.013 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:10.011+0000 7fe0c34d1640 1 -- 172.21.15.3:0/1381094286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_bytes", "val": "0"} v 0) -- 0x7fe0bc0697f0 con 0x7fe0bc09da20 2024-09-22T04:06:10.894 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:10.892+0000 7fe0b1ffb640 1 -- 172.21.15.3:0/1381094286 <== mon.0 v2:172.21.15.3:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 v729) ==== 217+0+0 (crc 0 0 0) 0x7fe0b80109c0 con 0x7fe0bc09da20 2024-09-22T04:06:10.894 INFO:tasks.workunit.client.0.smithi003.stderr:set-quota max_bytes = 0 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 2024-09-22T04:06:10.894 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:10.893+0000 7fe0b1ffb640 1 -- 172.21.15.3:0/1381094286 <== mon.0 v2:172.21.15.3:3300/0 10 ==== osd_map(729..729 src has 590..729) ==== 628+0+0 (crc 0 0 0) 0x7fe0b80143b0 con 0x7fe0bc09da20 2024-09-22T04:06:10.894 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:10.893+0000 7fe0b1ffb640 1 -- 172.21.15.3:0/1381094286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=730}) -- 0x7fe0a0046670 con 0x7fe0bc09da20 2024-09-22T04:06:10.898 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:10.897+0000 7fe0c34d1640 1 -- 172.21.15.3:0/1381094286 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe0a003ead0 msgr2=0x7fe0a0040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:10.898 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:10.897+0000 7fe0c34d1640 1 --2- 172.21.15.3:0/1381094286 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe0a003ead0 0x7fe0a0040fe0 crc :-1 s=READY pgs=3865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:10.898 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:10.897+0000 7fe0c34d1640 1 -- 172.21.15.3:0/1381094286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc09da20 msgr2=0x7fe0bc1208a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:10.898 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:10.897+0000 7fe0c34d1640 1 --2- 172.21.15.3:0/1381094286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc09da20 0x7fe0bc1208a0 crc :-1 s=READY pgs=12992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:10.898 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:10.897+0000 7fe0c34d1640 1 -- 172.21.15.3:0/1381094286 shutdown_connections 2024-09-22T04:06:10.899 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:10.897+0000 7fe0c34d1640 1 --2- 172.21.15.3:0/1381094286 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe0a003ead0 0x7fe0a0040fe0 unknown :-1 s=CLOSED pgs=3865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:10.899 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:10.897+0000 7fe0c34d1640 1 --2- 172.21.15.3:0/1381094286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe0bc09da20 0x7fe0bc1208a0 unknown :-1 s=CLOSED pgs=12992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:10.899 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:10.897+0000 7fe0c34d1640 1 -- 172.21.15.3:0/1381094286 >> 172.21.15.3:0/1381094286 conn(0x7fe0bc0a6d10 msgr2=0x7fe0bc0a4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:10.899 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:10.897+0000 7fe0c34d1640 1 -- 172.21.15.3:0/1381094286 shutdown_connections 2024-09-22T04:06:10.899 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:10.897+0000 7fe0c34d1640 1 -- 172.21.15.3:0/1381094286 wait complete. 2024-09-22T04:06:10.917 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool set-quota 7c1c124a-aa8e-411d-9731-1f1389755458 max_objects 0 2024-09-22T04:06:11.034 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.024+0000 7f806e0d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:11.035 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.034+0000 7f806e0d1640 1 Processor -- start 2024-09-22T04:06:11.035 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.034+0000 7f806e0d1640 1 -- start start 2024-09-22T04:06:11.037 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.035+0000 7f806e0d1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f80681364c0 0x7f80681368d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:11.037 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.036+0000 7f806e0d1640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80680a16a0 con 0x7f8068136e10 2024-09-22T04:06:11.038 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.036+0000 7f806e0d1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f80680a0a00 con 0x7f80681364c0 2024-09-22T04:06:11.038 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.036+0000 7f80677fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f80681364c0 0x7f80681368d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:11.038 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.036+0000 7f8066ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8068136e10 0x7f806809f870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49336/0 (socket says 172.21.15.3:49336) 2024-09-22T04:06:11.038 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.036+0000 7f8066ffd640 1 -- 172.21.15.3:0/3370158248 learned_addr learned my addr 172.21.15.3:0/3370158248 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T04:06:11.038 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.036+0000 7f80677fe640 1 -- 172.21.15.3:0/3370158248 >> v1:172.21.15.3:6789/0 conn(0x7f8068136e10 legacy=0x7f806809f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:11.038 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.036+0000 7f80677fe640 1 -- 172.21.15.3:0/3370158248 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80680a13b0 con 0x7f80681364c0 2024-09-22T04:06:11.038 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.036+0000 7f80667fc640 1 -- 172.21.15.3:0/3370158248 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3309914025 0 0) 0x7f80680a16a0 con 0x7f8068136e10 2024-09-22T04:06:11.038 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.036+0000 7f80677fe640 1 --2- 172.21.15.3:0/3370158248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80681364c0 0x7f80681368d0 crc :-1 s=READY pgs=12994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9535bcc3a122ff9b server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:11.038 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.036+0000 7f80667fc640 1 -- 172.21.15.3:0/3370158248 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8058009d90 con 0x7f80681364c0 2024-09-22T04:06:11.038 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.036+0000 7f80667fc640 1 -- 172.21.15.3:0/3370158248 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f805800e230 con 0x7f80681364c0 2024-09-22T04:06:11.038 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.036+0000 7f80667fc640 1 -- 172.21.15.3:0/3370158248 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f805800e4d0 con 0x7f80681364c0 2024-09-22T04:06:11.039 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.037+0000 7f806e0d1640 1 -- 172.21.15.3:0/3370158248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80681364c0 msgr2=0x7f80681368d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:11.039 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.037+0000 7f806e0d1640 1 --2- 172.21.15.3:0/3370158248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80681364c0 0x7f80681368d0 crc :-1 s=READY pgs=12994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:11.039 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.037+0000 7f806e0d1640 1 -- 172.21.15.3:0/3370158248 shutdown_connections 2024-09-22T04:06:11.040 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.037+0000 7f806e0d1640 1 --2- 172.21.15.3:0/3370158248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80681364c0 0x7f80681368d0 unknown :-1 s=CLOSED pgs=12994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:11.040 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.037+0000 7f806e0d1640 1 -- 172.21.15.3:0/3370158248 >> 172.21.15.3:0/3370158248 conn(0x7f80680a6270 msgr2=0x7f80680a6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:11.040 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.037+0000 7f806e0d1640 1 -- 172.21.15.3:0/3370158248 shutdown_connections 2024-09-22T04:06:11.040 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.037+0000 7f806e0d1640 1 -- 172.21.15.3:0/3370158248 wait complete. 2024-09-22T04:06:11.040 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.037+0000 7f806e0d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:11.040 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.039+0000 7f806e0d1640 1 Processor -- start 2024-09-22T04:06:11.040 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.039+0000 7f806e0d1640 1 -- start start 2024-09-22T04:06:11.043 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.041+0000 7f806e0d1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80681364c0 0x7f80681ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:11.043 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.041+0000 7f806e0d1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8068136100 con 0x7f80681364c0 2024-09-22T04:06:11.043 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.041+0000 7f80677fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80681364c0 0x7f80681ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:11.043 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.041+0000 7f80677fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80681364c0 0x7f80681ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36442/0 (socket says 172.21.15.3:36442) 2024-09-22T04:06:11.043 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.041+0000 7f80677fe640 1 -- 172.21.15.3:0/3256938455 learned_addr learned my addr 172.21.15.3:0/3256938455 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:11.043 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.041+0000 7f80677fe640 1 -- 172.21.15.3:0/3256938455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80680a4210 con 0x7f80681364c0 2024-09-22T04:06:11.043 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.041+0000 7f80677fe640 1 --2- 172.21.15.3:0/3256938455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80681364c0 0x7f80681ca850 crc :-1 s=READY pgs=12995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:11.044 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.042+0000 7f804ffff640 1 -- 172.21.15.3:0/3256938455 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8058013610 con 0x7f80681364c0 2024-09-22T04:06:11.044 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.042+0000 7f804ffff640 1 -- 172.21.15.3:0/3256938455 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f8058013770 con 0x7f80681364c0 2024-09-22T04:06:11.044 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.042+0000 7f806e0d1640 1 -- 172.21.15.3:0/3256938455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8068132260 con 0x7f80681364c0 2024-09-22T04:06:11.044 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.042+0000 7f804ffff640 1 -- 172.21.15.3:0/3256938455 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f8058013a10 con 0x7f80681364c0 2024-09-22T04:06:11.044 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.042+0000 7f806e0d1640 1 -- 172.21.15.3:0/3256938455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80681343c0 con 0x7f80681364c0 2024-09-22T04:06:11.045 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.043+0000 7f806e0d1640 1 -- 172.21.15.3:0/3256938455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80680a2230 con 0x7f80681364c0 2024-09-22T04:06:11.045 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.043+0000 7f804ffff640 1 -- 172.21.15.3:0/3256938455 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f8058013bc0 con 0x7f80681364c0 2024-09-22T04:06:11.045 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.043+0000 7f804ffff640 1 --2- 172.21.15.3:0/3256938455 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f805403ead0 0x7f8054040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:11.045 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.043+0000 7f804ffff640 1 -- 172.21.15.3:0/3256938455 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(729..729 src has 590..729) ==== 4207+0+0 (crc 0 0 0) 0x7f8058018040 con 0x7f80681364c0 2024-09-22T04:06:11.045 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.043+0000 7f804ffff640 1 -- 172.21.15.3:0/3256938455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=730}) -- 0x7f8054045500 con 0x7f80681364c0 2024-09-22T04:06:11.045 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.044+0000 7f8066ffd640 1 --2- 172.21.15.3:0/3256938455 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f805403ead0 0x7f8054040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:11.045 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.044+0000 7f8066ffd640 1 --2- 172.21.15.3:0/3256938455 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f805403ead0 0x7f8054040fe0 crc :-1 s=READY pgs=3866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:11.049 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.048+0000 7f804ffff640 1 -- 172.21.15.3:0/3256938455 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f8058011570 con 0x7f80681364c0 2024-09-22T04:06:11.241 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.240+0000 7f806e0d1640 1 -- 172.21.15.3:0/3256938455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_objects", "val": "0"} v 0) -- 0x7f8068050fc0 con 0x7f80681364c0 2024-09-22T04:06:11.903 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.902+0000 7f804ffff640 1 -- 172.21.15.3:0/3256938455 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 v730) ==== 221+0+0 (crc 0 0 0) 0x7f80580080b0 con 0x7f80681364c0 2024-09-22T04:06:11.903 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.902+0000 7f804ffff640 1 -- 172.21.15.3:0/3256938455 <== mon.0 v2:172.21.15.3:3300/0 8 ==== osd_map(730..730 src has 590..730) ==== 628+0+0 (crc 0 0 0) 0x7f8058012e30 con 0x7f80681364c0 2024-09-22T04:06:11.903 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:11.902+0000 7f804ffff640 1 -- 172.21.15.3:0/3256938455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=731}) -- 0x7f8054046480 con 0x7f80681364c0 2024-09-22T04:06:12.050 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:12.048+0000 7f806e0d1640 1 -- 172.21.15.3:0/3256938455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_objects", "val": "0"} v 0) -- 0x7f8068136de0 con 0x7f80681364c0 2024-09-22T04:06:12.904 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:12.903+0000 7f804ffff640 1 -- 172.21.15.3:0/3256938455 <== mon.0 v2:172.21.15.3:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 v731) ==== 221+0+0 (crc 0 0 0) 0x7f8058018bf0 con 0x7f80681364c0 2024-09-22T04:06:12.905 INFO:tasks.workunit.client.0.smithi003.stderr:set-quota max_objects = 0 for pool 7c1c124a-aa8e-411d-9731-1f1389755458 2024-09-22T04:06:12.905 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:12.903+0000 7f804ffff640 1 -- 172.21.15.3:0/3256938455 <== mon.0 v2:172.21.15.3:3300/0 10 ==== osd_map(731..731 src has 590..731) ==== 628+0+0 (crc 0 0 0) 0x7f8058017190 con 0x7f80681364c0 2024-09-22T04:06:12.905 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:12.903+0000 7f804ffff640 1 -- 172.21.15.3:0/3256938455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=732}) -- 0x7f80540465a0 con 0x7f80681364c0 2024-09-22T04:06:12.909 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:12.908+0000 7f806e0d1640 1 -- 172.21.15.3:0/3256938455 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f805403ead0 msgr2=0x7f8054040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:12.909 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:12.908+0000 7f806e0d1640 1 --2- 172.21.15.3:0/3256938455 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f805403ead0 0x7f8054040fe0 crc :-1 s=READY pgs=3866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:12.909 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:12.908+0000 7f806e0d1640 1 -- 172.21.15.3:0/3256938455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80681364c0 msgr2=0x7f80681ca850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:12.910 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:12.908+0000 7f806e0d1640 1 --2- 172.21.15.3:0/3256938455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80681364c0 0x7f80681ca850 crc :-1 s=READY pgs=12995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:12.910 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:12.908+0000 7f806e0d1640 1 -- 172.21.15.3:0/3256938455 shutdown_connections 2024-09-22T04:06:12.910 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:12.908+0000 7f806e0d1640 1 --2- 172.21.15.3:0/3256938455 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f805403ead0 0x7f8054040fe0 unknown :-1 s=CLOSED pgs=3866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:12.910 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:12.908+0000 7f806e0d1640 1 --2- 172.21.15.3:0/3256938455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f80681364c0 0x7f80681ca850 unknown :-1 s=CLOSED pgs=12995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:12.910 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:12.908+0000 7f806e0d1640 1 -- 172.21.15.3:0/3256938455 >> 172.21.15.3:0/3256938455 conn(0x7f80680a6270 msgr2=0x7f80680714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:12.910 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:12.908+0000 7f806e0d1640 1 -- 172.21.15.3:0/3256938455 shutdown_connections 2024-09-22T04:06:12.910 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:12.908+0000 7f806e0d1640 1 -- 172.21.15.3:0/3256938455 wait complete. 2024-09-22T04:06:12.928 INFO:tasks.workunit.client.0.smithi003.stderr:+ sleep 30 2024-09-22T04:06:42.929 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool delete 7c1c124a-aa8e-411d-9731-1f1389755458 7c1c124a-aa8e-411d-9731-1f1389755458 --yes-i-really-really-mean-it 2024-09-22T04:06:43.040 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.029+0000 7fa053d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:43.041 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.040+0000 7fa053d91640 1 Processor -- start 2024-09-22T04:06:43.041 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.040+0000 7fa053d91640 1 -- start start 2024-09-22T04:06:43.043 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.041+0000 7fa053d91640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa04c136d40 0x7fa04c139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:43.044 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.041+0000 7fa053d91640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa04c09f530 con 0x7fa04c130720 2024-09-22T04:06:43.044 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.041+0000 7fa053d91640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa04c09f2d0 con 0x7fa04c136d40 2024-09-22T04:06:43.044 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.042+0000 7fa051b2d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa04c130720 0x7fa04c130b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45252/0 (socket says 172.21.15.3:45252) 2024-09-22T04:06:43.044 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.042+0000 7fa051b2d640 1 -- 172.21.15.3:0/2301441723 learned_addr learned my addr 172.21.15.3:0/2301441723 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T04:06:43.044 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.042+0000 7fa05132c640 1 --2- 172.21.15.3:0/2301441723 >> v2:172.21.15.3:3300/0 conn(0x7fa04c136d40 0x7fa04c139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:43.044 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.042+0000 7fa050b2b640 1 -- 172.21.15.3:0/2301441723 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 347816636 0 0) 0x7fa04c09f530 con 0x7fa04c130720 2024-09-22T04:06:43.044 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.042+0000 7fa050b2b640 1 -- 172.21.15.3:0/2301441723 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa038003580 con 0x7fa04c130720 2024-09-22T04:06:43.044 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.042+0000 7fa05132c640 1 -- 172.21.15.3:0/2301441723 >> v1:172.21.15.3:6789/0 conn(0x7fa04c130720 legacy=0x7fa04c130b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:43.044 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.042+0000 7fa05132c640 1 -- 172.21.15.3:0/2301441723 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c06fd30 con 0x7fa04c136d40 2024-09-22T04:06:43.044 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.042+0000 7fa05132c640 1 --2- 172.21.15.3:0/2301441723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa04c136d40 0x7fa04c139160 crc :-1 s=READY pgs=12997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981ccfd836b245a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:43.044 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.042+0000 7fa050b2b640 1 -- 172.21.15.3:0/2301441723 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa03c009d90 con 0x7fa04c136d40 2024-09-22T04:06:43.044 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.042+0000 7fa050b2b640 1 -- 172.21.15.3:0/2301441723 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa03c00e230 con 0x7fa04c136d40 2024-09-22T04:06:43.045 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.042+0000 7fa050b2b640 1 -- 172.21.15.3:0/2301441723 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa03c00e4d0 con 0x7fa04c136d40 2024-09-22T04:06:43.045 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.043+0000 7fa053d91640 1 -- 172.21.15.3:0/2301441723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa04c136d40 msgr2=0x7fa04c139160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:43.045 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.043+0000 7fa053d91640 1 --2- 172.21.15.3:0/2301441723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa04c136d40 0x7fa04c139160 crc :-1 s=READY pgs=12997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:43.045 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.043+0000 7fa053d91640 1 -- 172.21.15.3:0/2301441723 shutdown_connections 2024-09-22T04:06:43.045 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.043+0000 7fa053d91640 1 --2- 172.21.15.3:0/2301441723 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa04c136d40 0x7fa04c139160 unknown :-1 s=CLOSED pgs=12997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:43.045 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.043+0000 7fa053d91640 1 -- 172.21.15.3:0/2301441723 >> 172.21.15.3:0/2301441723 conn(0x7fa04c0a20c0 msgr2=0x7fa04c0a24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:43.046 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.043+0000 7fa053d91640 1 -- 172.21.15.3:0/2301441723 shutdown_connections 2024-09-22T04:06:43.046 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.043+0000 7fa053d91640 1 -- 172.21.15.3:0/2301441723 wait complete. 2024-09-22T04:06:43.046 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.044+0000 7fa053d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:43.046 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.045+0000 7fa053d91640 1 Processor -- start 2024-09-22T04:06:43.046 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.045+0000 7fa053d91640 1 -- start start 2024-09-22T04:06:43.048 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.047+0000 7fa053d91640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa04c130720 0x7fa04c12c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:43.048 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.047+0000 7fa053d91640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c0a1350 con 0x7fa04c130720 2024-09-22T04:06:43.048 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.047+0000 7fa051b2d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa04c130720 0x7fa04c12c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:43.049 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.047+0000 7fa051b2d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa04c130720 0x7fa04c12c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34404/0 (socket says 172.21.15.3:34404) 2024-09-22T04:06:43.049 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.047+0000 7fa051b2d640 1 -- 172.21.15.3:0/2762582196 learned_addr learned my addr 172.21.15.3:0/2762582196 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:43.049 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.047+0000 7fa051b2d640 1 -- 172.21.15.3:0/2762582196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c1304f0 con 0x7fa04c130720 2024-09-22T04:06:43.049 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.048+0000 7fa051b2d640 1 --2- 172.21.15.3:0/2762582196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa04c130720 0x7fa04c12c600 crc :-1 s=READY pgs=12998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:43.050 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.048+0000 7fa0427fc640 1 -- 172.21.15.3:0/2762582196 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa048005a60 con 0x7fa04c130720 2024-09-22T04:06:43.050 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.048+0000 7fa0427fc640 1 -- 172.21.15.3:0/2762582196 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa048005bc0 con 0x7fa04c130720 2024-09-22T04:06:43.050 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.048+0000 7fa053d91640 1 -- 172.21.15.3:0/2762582196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04c0748e0 con 0x7fa04c130720 2024-09-22T04:06:43.050 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.048+0000 7fa0427fc640 1 -- 172.21.15.3:0/2762582196 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa048005e60 con 0x7fa04c130720 2024-09-22T04:06:43.050 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.048+0000 7fa053d91640 1 -- 172.21.15.3:0/2762582196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04c12d6b0 con 0x7fa04c130720 2024-09-22T04:06:43.050 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.049+0000 7fa053d91640 1 -- 172.21.15.3:0/2762582196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa04c12d940 con 0x7fa04c130720 2024-09-22T04:06:43.050 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.049+0000 7fa0427fc640 1 -- 172.21.15.3:0/2762582196 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fa0480070b0 con 0x7fa04c130720 2024-09-22T04:06:43.051 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.049+0000 7fa0427fc640 1 --2- 172.21.15.3:0/2762582196 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa03003eb00 0x7fa030041010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:43.051 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.049+0000 7fa0427fc640 1 -- 172.21.15.3:0/2762582196 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(731..731 src has 590..731) ==== 4207+0+0 (crc 0 0 0) 0x7fa04804fde0 con 0x7fa04c130720 2024-09-22T04:06:43.051 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.049+0000 7fa0427fc640 1 -- 172.21.15.3:0/2762582196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=732}) -- 0x7fa030045530 con 0x7fa04c130720 2024-09-22T04:06:43.051 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.050+0000 7fa05132c640 1 --2- 172.21.15.3:0/2762582196 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa03003eb00 0x7fa030041010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:43.051 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.050+0000 7fa05132c640 1 --2- 172.21.15.3:0/2762582196 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa03003eb00 0x7fa030041010 crc :-1 s=READY pgs=3867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:43.055 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.054+0000 7fa0427fc640 1 -- 172.21.15.3:0/2762582196 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7fa04800f230 con 0x7fa04c130720 2024-09-22T04:06:43.245 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.244+0000 7fa053d91640 1 -- 172.21.15.3:0/2762582196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "pool2": "7c1c124a-aa8e-411d-9731-1f1389755458", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa04c130b30 con 0x7fa04c130720 2024-09-22T04:06:43.421 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.420+0000 7fa0427fc640 1 -- 172.21.15.3:0/2762582196 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "pool2": "7c1c124a-aa8e-411d-9731-1f1389755458", "yes_i_really_really_mean_it": true}]=0 pool '7c1c124a-aa8e-411d-9731-1f1389755458' removed v732) ==== 248+0+0 (crc 0 0 0) 0x7fa04800d660 con 0x7fa04c130720 2024-09-22T04:06:43.421 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.420+0000 7fa0427fc640 1 -- 172.21.15.3:0/2762582196 <== mon.0 v2:172.21.15.3:3300/0 8 ==== osd_map(732..732 src has 590..732) ==== 296+0+0 (crc 0 0 0) 0x7fa04800da00 con 0x7fa04c130720 2024-09-22T04:06:43.421 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.420+0000 7fa0427fc640 1 -- 172.21.15.3:0/2762582196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=733}) -- 0x7fa0300461b0 con 0x7fa04c130720 2024-09-22T04:06:43.563 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.562+0000 7fa053d91640 1 -- 172.21.15.3:0/2762582196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "pool2": "7c1c124a-aa8e-411d-9731-1f1389755458", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa04c12db20 con 0x7fa04c130720 2024-09-22T04:06:43.563 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.562+0000 7fa0427fc640 1 -- 172.21.15.3:0/2762582196 <== mon.0 v2:172.21.15.3:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "7c1c124a-aa8e-411d-9731-1f1389755458", "pool2": "7c1c124a-aa8e-411d-9731-1f1389755458", "yes_i_really_really_mean_it": true}]=0 pool '7c1c124a-aa8e-411d-9731-1f1389755458' does not exist v732) ==== 255+0+0 (crc 0 0 0) 0x7fa048051040 con 0x7fa04c130720 2024-09-22T04:06:43.563 INFO:tasks.workunit.client.0.smithi003.stderr:pool '7c1c124a-aa8e-411d-9731-1f1389755458' does not exist 2024-09-22T04:06:43.566 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.565+0000 7fa053d91640 1 -- 172.21.15.3:0/2762582196 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa03003eb00 msgr2=0x7fa030041010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:43.566 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.565+0000 7fa053d91640 1 --2- 172.21.15.3:0/2762582196 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa03003eb00 0x7fa030041010 crc :-1 s=READY pgs=3867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:43.566 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.565+0000 7fa053d91640 1 -- 172.21.15.3:0/2762582196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa04c130720 msgr2=0x7fa04c12c600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:43.566 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.565+0000 7fa053d91640 1 --2- 172.21.15.3:0/2762582196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa04c130720 0x7fa04c12c600 crc :-1 s=READY pgs=12998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:43.567 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.565+0000 7fa053d91640 1 -- 172.21.15.3:0/2762582196 shutdown_connections 2024-09-22T04:06:43.567 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.565+0000 7fa053d91640 1 --2- 172.21.15.3:0/2762582196 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa03003eb00 0x7fa030041010 unknown :-1 s=CLOSED pgs=3867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:43.567 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.565+0000 7fa053d91640 1 --2- 172.21.15.3:0/2762582196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa04c130720 0x7fa04c12c600 unknown :-1 s=CLOSED pgs=12998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:43.567 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.565+0000 7fa053d91640 1 -- 172.21.15.3:0/2762582196 >> 172.21.15.3:0/2762582196 conn(0x7fa04c0a20c0 msgr2=0x7fa04c134b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:43.567 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.565+0000 7fa053d91640 1 -- 172.21.15.3:0/2762582196 shutdown_connections 2024-09-22T04:06:43.567 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.565+0000 7fa053d91640 1 -- 172.21.15.3:0/2762582196 wait complete. 2024-09-22T04:06:43.578 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool delete dd65dfbd-105b-4a15-8dc2-717b17ae7efb dd65dfbd-105b-4a15-8dc2-717b17ae7efb --yes-i-really-really-mean-it 2024-09-22T04:06:43.682 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.671+0000 7f93838ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:43.683 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.682+0000 7f93838ec640 1 Processor -- start 2024-09-22T04:06:43.683 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.682+0000 7f93838ec640 1 -- start start 2024-09-22T04:06:43.685 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.684+0000 7f93838ec640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f937c136fe0 0x7f937c13b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:43.685 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.684+0000 7f93838ec640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f937c0a16a0 con 0x7f937c1364c0 2024-09-22T04:06:43.685 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.684+0000 7f93838ec640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f937c0a0a00 con 0x7f937c136fe0 2024-09-22T04:06:43.685 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.684+0000 7f9380e87640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f937c136fe0 0x7f937c13b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:43.686 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.684+0000 7f9380e87640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f937c136fe0 0x7f937c13b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34410/0 (socket says 172.21.15.3:34410) 2024-09-22T04:06:43.686 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.684+0000 7f9381688640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f937c1364c0 0x7f937c1368d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45264/0 (socket says 172.21.15.3:45264) 2024-09-22T04:06:43.686 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.684+0000 7f9380e87640 1 -- 172.21.15.3:0/4166950051 learned_addr learned my addr 172.21.15.3:0/4166950051 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:43.687 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.684+0000 7f9380e87640 1 -- 172.21.15.3:0/4166950051 >> v1:172.21.15.3:6789/0 conn(0x7f937c1364c0 legacy=0x7f937c1368d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:43.687 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.684+0000 7f9380e87640 1 -- 172.21.15.3:0/4166950051 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c0a13b0 con 0x7f937c136fe0 2024-09-22T04:06:43.687 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.684+0000 7f9373fff640 1 -- 172.21.15.3:0/4166950051 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1393312166 0 0) 0x7f937c0a16a0 con 0x7f937c1364c0 2024-09-22T04:06:43.687 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.685+0000 7f9380e87640 1 --2- 172.21.15.3:0/4166950051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c136fe0 0x7f937c13b700 crc :-1 s=READY pgs=13000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d57ccc1692aa15f server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:43.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.685+0000 7f9373fff640 1 -- 172.21.15.3:0/4166950051 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f936c009e50 con 0x7f937c136fe0 2024-09-22T04:06:43.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.685+0000 7f9373fff640 1 -- 172.21.15.3:0/4166950051 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f936c00e230 con 0x7f937c136fe0 2024-09-22T04:06:43.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.685+0000 7f9373fff640 1 -- 172.21.15.3:0/4166950051 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f936c00e4d0 con 0x7f937c136fe0 2024-09-22T04:06:43.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.686+0000 7f93838ec640 1 -- 172.21.15.3:0/4166950051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c136fe0 msgr2=0x7f937c13b700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:43.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.686+0000 7f93838ec640 1 --2- 172.21.15.3:0/4166950051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c136fe0 0x7f937c13b700 crc :-1 s=READY pgs=13000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:43.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.686+0000 7f93838ec640 1 -- 172.21.15.3:0/4166950051 shutdown_connections 2024-09-22T04:06:43.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.686+0000 7f93838ec640 1 --2- 172.21.15.3:0/4166950051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c136fe0 0x7f937c13b700 unknown :-1 s=CLOSED pgs=13000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:43.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.686+0000 7f93838ec640 1 -- 172.21.15.3:0/4166950051 >> 172.21.15.3:0/4166950051 conn(0x7f937c0a6270 msgr2=0x7f937c0a6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:43.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.686+0000 7f93838ec640 1 -- 172.21.15.3:0/4166950051 shutdown_connections 2024-09-22T04:06:43.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.686+0000 7f93838ec640 1 -- 172.21.15.3:0/4166950051 wait complete. 2024-09-22T04:06:43.688 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.686+0000 7f93838ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:43.689 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.688+0000 7f93838ec640 1 Processor -- start 2024-09-22T04:06:43.689 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.688+0000 7f93838ec640 1 -- start start 2024-09-22T04:06:43.691 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.689+0000 7f93838ec640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c1364c0 0x7f937c09e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:43.692 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.690+0000 7f93838ec640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f937c133be0 con 0x7f937c1364c0 2024-09-22T04:06:43.692 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.690+0000 7f9381688640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c1364c0 0x7f937c09e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:43.692 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.690+0000 7f9381688640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c1364c0 0x7f937c09e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34414/0 (socket says 172.21.15.3:34414) 2024-09-22T04:06:43.692 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.690+0000 7f9381688640 1 -- 172.21.15.3:0/1169963254 learned_addr learned my addr 172.21.15.3:0/1169963254 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:43.692 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.690+0000 7f9381688640 1 -- 172.21.15.3:0/1169963254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c136120 con 0x7f937c1364c0 2024-09-22T04:06:43.692 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.690+0000 7f9381688640 1 --2- 172.21.15.3:0/1169963254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c1364c0 0x7f937c09e850 crc :-1 s=READY pgs=13001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:43.692 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.690+0000 7f9371ffb640 1 -- 172.21.15.3:0/1169963254 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9378004b80 con 0x7f937c1364c0 2024-09-22T04:06:43.692 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.690+0000 7f9371ffb640 1 -- 172.21.15.3:0/1169963254 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f9378004ce0 con 0x7f937c1364c0 2024-09-22T04:06:43.692 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.690+0000 7f9371ffb640 1 -- 172.21.15.3:0/1169963254 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f9378007120 con 0x7f937c1364c0 2024-09-22T04:06:43.692 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.690+0000 7f93838ec640 1 -- 172.21.15.3:0/1169963254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937c132430 con 0x7f937c1364c0 2024-09-22T04:06:43.692 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.691+0000 7f93838ec640 1 -- 172.21.15.3:0/1169963254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f937c0a42e0 con 0x7f937c1364c0 2024-09-22T04:06:43.693 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.692+0000 7f9371ffb640 1 -- 172.21.15.3:0/1169963254 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f93780072d0 con 0x7f937c1364c0 2024-09-22T04:06:43.693 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.692+0000 7f93838ec640 1 -- 172.21.15.3:0/1169963254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f937c0a2230 con 0x7f937c1364c0 2024-09-22T04:06:43.693 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.692+0000 7f9371ffb640 1 --2- 172.21.15.3:0/1169963254 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f936003ead0 0x7f9360040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:43.694 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.692+0000 7f9371ffb640 1 -- 172.21.15.3:0/1169963254 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(732..732 src has 590..732) ==== 3819+0+0 (crc 0 0 0) 0x7f937804fd70 con 0x7f937c1364c0 2024-09-22T04:06:43.694 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.692+0000 7f9371ffb640 1 -- 172.21.15.3:0/1169963254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=733}) -- 0x7f9360044d60 con 0x7f937c1364c0 2024-09-22T04:06:43.694 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.693+0000 7f9380e87640 1 --2- 172.21.15.3:0/1169963254 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f936003ead0 0x7f9360040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:43.694 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.693+0000 7f9380e87640 1 --2- 172.21.15.3:0/1169963254 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f936003ead0 0x7f9360040fe0 crc :-1 s=READY pgs=3868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:43.699 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.698+0000 7f9371ffb640 1 -- 172.21.15.3:0/1169963254 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f937800f420 con 0x7f937c1364c0 2024-09-22T04:06:43.894 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:43.893+0000 7f93838ec640 1 -- 172.21.15.3:0/1169963254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "pool2": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "yes_i_really_really_mean_it": true} v 0) -- 0x7f937c050fc0 con 0x7f937c1364c0 2024-09-22T04:06:44.422 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:44.421+0000 7f9371ffb640 1 -- 172.21.15.3:0/1169963254 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "pool2": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "yes_i_really_really_mean_it": true}]=0 pool 'dd65dfbd-105b-4a15-8dc2-717b17ae7efb' removed v733) ==== 248+0+0 (crc 0 0 0) 0x7f937800a0c0 con 0x7f937c1364c0 2024-09-22T04:06:44.424 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:44.423+0000 7f9371ffb640 1 -- 172.21.15.3:0/1169963254 <== mon.0 v2:172.21.15.3:3300/0 8 ==== osd_map(733..733 src has 590..733) ==== 296+0+0 (crc 0 0 0) 0x7f9378021eb0 con 0x7f937c1364c0 2024-09-22T04:06:44.424 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:44.423+0000 7f9371ffb640 1 -- 172.21.15.3:0/1169963254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=734}) -- 0x7f9360045ab0 con 0x7f937c1364c0 2024-09-22T04:06:44.590 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:44.588+0000 7f93838ec640 1 -- 172.21.15.3:0/1169963254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "pool2": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "yes_i_really_really_mean_it": true} v 0) -- 0x7f937c074a10 con 0x7f937c1364c0 2024-09-22T04:06:44.590 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:44.589+0000 7f9371ffb640 1 -- 172.21.15.3:0/1169963254 <== mon.0 v2:172.21.15.3:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "pool2": "dd65dfbd-105b-4a15-8dc2-717b17ae7efb", "yes_i_really_really_mean_it": true}]=0 pool 'dd65dfbd-105b-4a15-8dc2-717b17ae7efb' does not exist v733) ==== 255+0+0 (crc 0 0 0) 0x7f937800a7b0 con 0x7f937c1364c0 2024-09-22T04:06:44.591 INFO:tasks.workunit.client.0.smithi003.stderr:pool 'dd65dfbd-105b-4a15-8dc2-717b17ae7efb' does not exist 2024-09-22T04:06:44.593 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:44.591+0000 7f93838ec640 1 -- 172.21.15.3:0/1169963254 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f936003ead0 msgr2=0x7f9360040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:44.593 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:44.591+0000 7f93838ec640 1 --2- 172.21.15.3:0/1169963254 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f936003ead0 0x7f9360040fe0 crc :-1 s=READY pgs=3868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:44.593 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:44.591+0000 7f93838ec640 1 -- 172.21.15.3:0/1169963254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c1364c0 msgr2=0x7f937c09e850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:44.593 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:44.591+0000 7f93838ec640 1 --2- 172.21.15.3:0/1169963254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c1364c0 0x7f937c09e850 crc :-1 s=READY pgs=13001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:44.594 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:44.591+0000 7f93838ec640 1 -- 172.21.15.3:0/1169963254 shutdown_connections 2024-09-22T04:06:44.594 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:44.591+0000 7f93838ec640 1 --2- 172.21.15.3:0/1169963254 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f936003ead0 0x7f9360040fe0 unknown :-1 s=CLOSED pgs=3868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:44.594 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:44.591+0000 7f93838ec640 1 --2- 172.21.15.3:0/1169963254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f937c1364c0 0x7f937c09e850 unknown :-1 s=CLOSED pgs=13001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:44.594 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:44.591+0000 7f93838ec640 1 -- 172.21.15.3:0/1169963254 >> 172.21.15.3:0/1169963254 conn(0x7f937c0a6270 msgr2=0x7f937c0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:44.594 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:44.591+0000 7f93838ec640 1 -- 172.21.15.3:0/1169963254 shutdown_connections 2024-09-22T04:06:44.594 INFO:tasks.workunit.client.0.smithi003.stderr:2024-09-22T04:06:44.591+0000 7f93838ec640 1 -- 172.21.15.3:0/1169963254 wait complete. 2024-09-22T04:06:44.604 INFO:tasks.workunit.client.0.smithi003.stderr:+ echo OK 2024-09-22T04:06:44.605 INFO:tasks.workunit.client.0.smithi003.stdout:OK 2024-09-22T04:06:44.606 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-22T04:06:44.606 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-22T04:06:44.674 INFO:tasks.workunit:Stopping ['rados/test.sh --crimson', 'rados/test_pool_quota.sh'] on client.0... 2024-09-22T04:06:44.674 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-22T04:06:45.033 DEBUG:teuthology.parallel:result is None 2024-09-22T04:06:45.034 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-22T04:06:45.059 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-22T04:06:45.059 DEBUG:teuthology.orchestra.run.smithi003:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-22T04:06:45.121 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-22T04:06:45.122 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-22T04:06:45.130 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-22T04:06:45.130 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T04:06:45.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.270+0000 7f2d2cefd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:45.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.275+0000 7f2d2cefd640 1 Processor -- start 2024-09-22T04:06:45.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.275+0000 7f2d2cefd640 1 -- start start 2024-09-22T04:06:45.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.277+0000 7f2d2cefd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2d2806e7b0 0x7f2d2806ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:45.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.277+0000 7f2d2cefd640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2809da70 con 0x7f2d2806f100 2024-09-22T04:06:45.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.277+0000 7f2d2cefd640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2d2809eb60 con 0x7f2d2806e7b0 2024-09-22T04:06:45.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.277+0000 7f2d2659c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2d2806e7b0 0x7f2d2806ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:45.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.277+0000 7f2d2659c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2d2806e7b0 0x7f2d2806ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34420/0 (socket says 172.21.15.3:34420) 2024-09-22T04:06:45.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.277+0000 7f2d25d9b640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2d2806f100 0x7f2d28133a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45278/0 (socket says 172.21.15.3:45278) 2024-09-22T04:06:45.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.277+0000 7f2d2659c640 1 -- 172.21.15.3:0/98140060 learned_addr learned my addr 172.21.15.3:0/98140060 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:45.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.277+0000 7f2d2659c640 1 -- 172.21.15.3:0/98140060 >> v1:172.21.15.3:6789/0 conn(0x7f2d2806f100 legacy=0x7f2d28133a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:45.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.277+0000 7f2d2659c640 1 -- 172.21.15.3:0/98140060 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2809ef00 con 0x7f2d2806e7b0 2024-09-22T04:06:45.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.277+0000 7f2d2559a640 1 -- 172.21.15.3:0/98140060 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1131959007 0 0) 0x7f2d2809da70 con 0x7f2d2806f100 2024-09-22T04:06:45.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.278+0000 7f2d2659c640 1 --2- 172.21.15.3:0/98140060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d2806e7b0 0x7f2d2806ebc0 crc :-1 s=READY pgs=13003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc69c67b1727e40 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:45.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.278+0000 7f2d2559a640 1 -- 172.21.15.3:0/98140060 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2d18009e50 con 0x7f2d2806e7b0 2024-09-22T04:06:45.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.278+0000 7f2d2559a640 1 -- 172.21.15.3:0/98140060 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2d1800e230 con 0x7f2d2806e7b0 2024-09-22T04:06:45.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.278+0000 7f2d2559a640 1 -- 172.21.15.3:0/98140060 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2d1800e4d0 con 0x7f2d2806e7b0 2024-09-22T04:06:45.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.278+0000 7f2d2cefd640 1 -- 172.21.15.3:0/98140060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d2806e7b0 msgr2=0x7f2d2806ebc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:45.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.278+0000 7f2d2cefd640 1 --2- 172.21.15.3:0/98140060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d2806e7b0 0x7f2d2806ebc0 crc :-1 s=READY pgs=13003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:45.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.279+0000 7f2d2cefd640 1 -- 172.21.15.3:0/98140060 shutdown_connections 2024-09-22T04:06:45.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.279+0000 7f2d2cefd640 1 --2- 172.21.15.3:0/98140060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d2806e7b0 0x7f2d2806ebc0 unknown :-1 s=CLOSED pgs=13003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:45.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.279+0000 7f2d2cefd640 1 -- 172.21.15.3:0/98140060 >> 172.21.15.3:0/98140060 conn(0x7f2d280a40f0 msgr2=0x7f2d280a4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:45.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.279+0000 7f2d2cefd640 1 -- 172.21.15.3:0/98140060 shutdown_connections 2024-09-22T04:06:45.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.279+0000 7f2d2cefd640 1 -- 172.21.15.3:0/98140060 wait complete. 2024-09-22T04:06:45.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.279+0000 7f2d2cefd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:45.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.281+0000 7f2d2cefd640 1 Processor -- start 2024-09-22T04:06:45.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.281+0000 7f2d2cefd640 1 -- start start 2024-09-22T04:06:45.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.282+0000 7f2d2cefd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d2806e7b0 0x7f2d28121e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:45.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.282+0000 7f2d2cefd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281238d0 con 0x7f2d2806e7b0 2024-09-22T04:06:45.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.282+0000 7f2d2659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d2806e7b0 0x7f2d28121e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:45.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.282+0000 7f2d2659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d2806e7b0 0x7f2d28121e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34430/0 (socket says 172.21.15.3:34430) 2024-09-22T04:06:45.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.282+0000 7f2d2659c640 1 -- 172.21.15.3:0/2315127792 learned_addr learned my addr 172.21.15.3:0/2315127792 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:45.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.283+0000 7f2d2659c640 1 -- 172.21.15.3:0/2315127792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d28121a70 con 0x7f2d2806e7b0 2024-09-22T04:06:45.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.283+0000 7f2d2659c640 1 --2- 172.21.15.3:0/2315127792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d2806e7b0 0x7f2d28121e30 crc :-1 s=READY pgs=13004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:45.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.283+0000 7f2d16ffd640 1 -- 172.21.15.3:0/2315127792 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2d1800edc0 con 0x7f2d2806e7b0 2024-09-22T04:06:45.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.283+0000 7f2d16ffd640 1 -- 172.21.15.3:0/2315127792 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f2d18015420 con 0x7f2d2806e7b0 2024-09-22T04:06:45.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.283+0000 7f2d2cefd640 1 -- 172.21.15.3:0/2315127792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d28121470 con 0x7f2d2806e7b0 2024-09-22T04:06:45.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.283+0000 7f2d16ffd640 1 -- 172.21.15.3:0/2315127792 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f2d180156c0 con 0x7f2d2806e7b0 2024-09-22T04:06:45.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.283+0000 7f2d2cefd640 1 -- 172.21.15.3:0/2315127792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d28123d60 con 0x7f2d2806e7b0 2024-09-22T04:06:45.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.284+0000 7f2d2cefd640 1 -- 172.21.15.3:0/2315127792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d280a4db0 con 0x7f2d2806e7b0 2024-09-22T04:06:45.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.284+0000 7f2d16ffd640 1 -- 172.21.15.3:0/2315127792 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f2d18015870 con 0x7f2d2806e7b0 2024-09-22T04:06:45.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.285+0000 7f2d16ffd640 1 --2- 172.21.15.3:0/2315127792 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f2d0803eb00 0x7f2d08041010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:45.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.285+0000 7f2d16ffd640 1 -- 172.21.15.3:0/2315127792 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(733..733 src has 590..733) ==== 3431+0+0 (crc 0 0 0) 0x7f2d180510e0 con 0x7f2d2806e7b0 2024-09-22T04:06:45.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.285+0000 7f2d25d9b640 1 --2- 172.21.15.3:0/2315127792 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f2d0803eb00 0x7f2d08041010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:45.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.286+0000 7f2d25d9b640 1 --2- 172.21.15.3:0/2315127792 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f2d0803eb00 0x7f2d08041010 crc :-1 s=READY pgs=3869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:45.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.290+0000 7f2d16ffd640 1 -- 172.21.15.3:0/2315127792 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f2d18010a00 con 0x7f2d2806e7b0 2024-09-22T04:06:45.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.474+0000 7f2d2cefd640 1 -- 172.21.15.3:0/2315127792 --> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d28121870 con 0x7f2d0803eb00 2024-09-22T04:06:45.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.478+0000 7f2d16ffd640 1 -- 172.21.15.3:0/2315127792 <== mgr.4099 v2:172.21.15.123:6800/551784226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+109430 (crc 0 0 0) 0x7f2d28121870 con 0x7f2d0803eb00 2024-09-22T04:06:45.480 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T04:06:45.480 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-22T04:06:45.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.481+0000 7f2d2cefd640 1 -- 172.21.15.3:0/2315127792 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f2d0803eb00 msgr2=0x7f2d08041010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:45.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.481+0000 7f2d2cefd640 1 --2- 172.21.15.3:0/2315127792 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f2d0803eb00 0x7f2d08041010 crc :-1 s=READY pgs=3869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.481+0000 7f2d2cefd640 1 -- 172.21.15.3:0/2315127792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d2806e7b0 msgr2=0x7f2d28121e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.481+0000 7f2d2cefd640 1 --2- 172.21.15.3:0/2315127792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d2806e7b0 0x7f2d28121e30 crc :-1 s=READY pgs=13004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.481+0000 7f2d2cefd640 1 -- 172.21.15.3:0/2315127792 shutdown_connections 2024-09-22T04:06:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.481+0000 7f2d2cefd640 1 --2- 172.21.15.3:0/2315127792 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f2d0803eb00 0x7f2d08041010 unknown :-1 s=CLOSED pgs=3869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.481+0000 7f2d2cefd640 1 --2- 172.21.15.3:0/2315127792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d2806e7b0 0x7f2d28121e30 unknown :-1 s=CLOSED pgs=13004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.481+0000 7f2d2cefd640 1 -- 172.21.15.3:0/2315127792 >> 172.21.15.3:0/2315127792 conn(0x7f2d280a40f0 msgr2=0x7f2d280a0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.481+0000 7f2d2cefd640 1 -- 172.21.15.3:0/2315127792 shutdown_connections 2024-09-22T04:06:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.481+0000 7f2d2cefd640 1 -- 172.21.15.3:0/2315127792 wait complete. 2024-09-22T04:06:45.493 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":1324,"stamp":"2024-09-22T04:06:44.424778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590392,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":52,"num_write":88,"num_write_kb":1115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62,"ondisk_log_size":62,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":23472,"kb_used_data":23472,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209167440,"statfs":{"total":214211493888,"available":214187458560,"internally_reserved":0,"allocated":24035328,"data_stored":24035328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-321875,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-25,"num_write_kb":-325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.585809"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.941328+0000","last_change":"2024-09-22T03:52:13.461984+0000","last_active":"2024-09-22T04:06:12.941328+0000","last_peered":"2024-09-22T04:06:12.941328+0000","last_clean":"2024-09-22T04:06:12.941328+0000","last_became_active":"2024-09-22T03:52:13.436104+0000","last_became_peered":"2024-09-22T03:52:13.436104+0000","last_unstale":"2024-09-22T04:06:12.941328+0000","last_undegraded":"2024-09-22T04:06:12.941328+0000","last_fullsized":"2024-09-22T04:06:12.941328+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.950121+0000","last_change":"2024-09-22T03:52:13.517533+0000","last_active":"2024-09-22T04:06:12.950121+0000","last_peered":"2024-09-22T04:06:12.950121+0000","last_clean":"2024-09-22T04:06:12.950121+0000","last_became_active":"2024-09-22T03:52:13.485941+0000","last_became_peered":"2024-09-22T03:52:13.485941+0000","last_unstale":"2024-09-22T04:06:12.950121+0000","last_undegraded":"2024-09-22T04:06:12.950121+0000","last_fullsized":"2024-09-22T04:06:12.950121+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.939878+0000","last_change":"2024-09-22T03:52:13.441983+0000","last_active":"2024-09-22T04:06:12.939878+0000","last_peered":"2024-09-22T04:06:12.939878+0000","last_clean":"2024-09-22T04:06:12.939878+0000","last_became_active":"2024-09-22T03:52:13.435054+0000","last_became_peered":"2024-09-22T03:52:13.435054+0000","last_unstale":"2024-09-22T04:06:12.939878+0000","last_undegraded":"2024-09-22T04:06:12.939878+0000","last_fullsized":"2024-09-22T04:06:12.939878+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.938208+0000","last_change":"2024-09-22T03:52:13.475862+0000","last_active":"2024-09-22T04:06:12.938208+0000","last_peered":"2024-09-22T04:06:12.938208+0000","last_clean":"2024-09-22T04:06:12.938208+0000","last_became_active":"2024-09-22T03:52:13.424125+0000","last_became_peered":"2024-09-22T03:52:13.424125+0000","last_unstale":"2024-09-22T04:06:12.938208+0000","last_undegraded":"2024-09-22T04:06:12.938208+0000","last_fullsized":"2024-09-22T04:06:12.938208+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.959684+0000","last_change":"2024-09-22T03:52:13.638357+0000","last_active":"2024-09-22T04:06:12.959684+0000","last_peered":"2024-09-22T04:06:12.959684+0000","last_clean":"2024-09-22T04:06:12.959684+0000","last_became_active":"2024-09-22T03:52:13.612276+0000","last_became_peered":"2024-09-22T03:52:13.612276+0000","last_unstale":"2024-09-22T04:06:12.959684+0000","last_undegraded":"2024-09-22T04:06:12.959684+0000","last_fullsized":"2024-09-22T04:06:12.959684+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.955663+0000","last_change":"2024-09-22T03:52:13.631942+0000","last_active":"2024-09-22T04:06:12.955663+0000","last_peered":"2024-09-22T04:06:12.955663+0000","last_clean":"2024-09-22T04:06:12.955663+0000","last_became_active":"2024-09-22T03:52:13.584449+0000","last_became_peered":"2024-09-22T03:52:13.584449+0000","last_unstale":"2024-09-22T04:06:12.955663+0000","last_undegraded":"2024-09-22T04:06:12.955663+0000","last_fullsized":"2024-09-22T04:06:12.955663+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.943719+0000","last_change":"2024-09-22T03:52:13.462393+0000","last_active":"2024-09-22T04:06:12.943719+0000","last_peered":"2024-09-22T04:06:12.943719+0000","last_clean":"2024-09-22T04:06:12.943719+0000","last_became_active":"2024-09-22T03:52:13.438920+0000","last_became_peered":"2024-09-22T03:52:13.438920+0000","last_unstale":"2024-09-22T04:06:12.943719+0000","last_undegraded":"2024-09-22T04:06:12.943719+0000","last_fullsized":"2024-09-22T04:06:12.943719+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.955610+0000","last_change":"2024-09-22T03:52:13.435067+0000","last_active":"2024-09-22T04:06:12.955610+0000","last_peered":"2024-09-22T04:06:12.955610+0000","last_clean":"2024-09-22T04:06:12.955610+0000","last_became_active":"2024-09-22T03:52:13.424399+0000","last_became_peered":"2024-09-22T03:52:13.424399+0000","last_unstale":"2024-09-22T04:06:12.955610+0000","last_undegraded":"2024-09-22T04:06:12.955610+0000","last_fullsized":"2024-09-22T04:06:12.955610+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.935572+0000","last_change":"2024-09-22T03:52:13.414909+0000","last_active":"2024-09-22T04:06:12.935572+0000","last_peered":"2024-09-22T04:06:12.935572+0000","last_clean":"2024-09-22T04:06:12.935572+0000","last_became_active":"2024-09-22T03:52:13.367950+0000","last_became_peered":"2024-09-22T03:52:13.367950+0000","last_unstale":"2024-09-22T04:06:12.935572+0000","last_undegraded":"2024-09-22T04:06:12.935572+0000","last_fullsized":"2024-09-22T04:06:12.935572+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.936631+0000","last_change":"2024-09-22T03:52:13.421431+0000","last_active":"2024-09-22T04:06:12.936631+0000","last_peered":"2024-09-22T04:06:12.936631+0000","last_clean":"2024-09-22T04:06:12.936631+0000","last_became_active":"2024-09-22T03:52:13.370886+0000","last_became_peered":"2024-09-22T03:52:13.370886+0000","last_unstale":"2024-09-22T04:06:12.936631+0000","last_undegraded":"2024-09-22T04:06:12.936631+0000","last_fullsized":"2024-09-22T04:06:12.936631+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.946060+0000","last_change":"2024-09-22T03:52:13.488521+0000","last_active":"2024-09-22T04:06:12.946060+0000","last_peered":"2024-09-22T04:06:12.946060+0000","last_clean":"2024-09-22T04:06:12.946060+0000","last_became_active":"2024-09-22T03:52:13.477618+0000","last_became_peered":"2024-09-22T03:52:13.477618+0000","last_unstale":"2024-09-22T04:06:12.946060+0000","last_undegraded":"2024-09-22T04:06:12.946060+0000","last_fullsized":"2024-09-22T04:06:12.946060+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.942381+0000","last_change":"2024-09-22T03:52:13.423108+0000","last_active":"2024-09-22T04:06:12.942381+0000","last_peered":"2024-09-22T04:06:12.942381+0000","last_clean":"2024-09-22T04:06:12.942381+0000","last_became_active":"2024-09-22T03:52:13.354826+0000","last_became_peered":"2024-09-22T03:52:13.354826+0000","last_unstale":"2024-09-22T04:06:12.942381+0000","last_undegraded":"2024-09-22T04:06:12.942381+0000","last_fullsized":"2024-09-22T04:06:12.942381+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.939191+0000","last_change":"2024-09-22T03:52:13.456498+0000","last_active":"2024-09-22T04:06:12.939191+0000","last_peered":"2024-09-22T04:06:12.939191+0000","last_clean":"2024-09-22T04:06:12.939191+0000","last_became_active":"2024-09-22T03:52:13.405184+0000","last_became_peered":"2024-09-22T03:52:13.405184+0000","last_unstale":"2024-09-22T04:06:12.939191+0000","last_undegraded":"2024-09-22T04:06:12.939191+0000","last_fullsized":"2024-09-22T04:06:12.939191+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.943494+0000","last_change":"2024-09-22T03:52:13.548435+0000","last_active":"2024-09-22T04:06:12.943494+0000","last_peered":"2024-09-22T04:06:12.943494+0000","last_clean":"2024-09-22T04:06:12.943494+0000","last_became_active":"2024-09-22T03:52:13.533286+0000","last_became_peered":"2024-09-22T03:52:13.533286+0000","last_unstale":"2024-09-22T04:06:12.943494+0000","last_undegraded":"2024-09-22T04:06:12.943494+0000","last_fullsized":"2024-09-22T04:06:12.943494+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.944076+0000","last_change":"2024-09-22T03:52:13.555314+0000","last_active":"2024-09-22T04:06:12.944076+0000","last_peered":"2024-09-22T04:06:12.944076+0000","last_clean":"2024-09-22T04:06:12.944076+0000","last_became_active":"2024-09-22T03:52:13.531887+0000","last_became_peered":"2024-09-22T03:52:13.531887+0000","last_unstale":"2024-09-22T04:06:12.944076+0000","last_undegraded":"2024-09-22T04:06:12.944076+0000","last_fullsized":"2024-09-22T04:06:12.944076+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.942361+0000","last_change":"2024-09-22T03:52:13.576998+0000","last_active":"2024-09-22T04:06:12.942361+0000","last_peered":"2024-09-22T04:06:12.942361+0000","last_clean":"2024-09-22T04:06:12.942361+0000","last_became_active":"2024-09-22T03:52:13.562231+0000","last_became_peered":"2024-09-22T03:52:13.562231+0000","last_unstale":"2024-09-22T04:06:12.942361+0000","last_undegraded":"2024-09-22T04:06:12.942361+0000","last_fullsized":"2024-09-22T04:06:12.942361+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.952801+0000","last_change":"2024-09-22T03:52:13.527047+0000","last_active":"2024-09-22T04:06:12.952801+0000","last_peered":"2024-09-22T04:06:12.952801+0000","last_clean":"2024-09-22T04:06:12.952801+0000","last_became_active":"2024-09-22T03:52:13.496766+0000","last_became_peered":"2024-09-22T03:52:13.496766+0000","last_unstale":"2024-09-22T04:06:12.952801+0000","last_undegraded":"2024-09-22T04:06:12.952801+0000","last_fullsized":"2024-09-22T04:06:12.952801+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.949322+0000","last_change":"2024-09-22T03:52:13.517430+0000","last_active":"2024-09-22T04:06:12.949322+0000","last_peered":"2024-09-22T04:06:12.949322+0000","last_clean":"2024-09-22T04:06:12.949322+0000","last_became_active":"2024-09-22T03:52:13.496886+0000","last_became_peered":"2024-09-22T03:52:13.496886+0000","last_unstale":"2024-09-22T04:06:12.949322+0000","last_undegraded":"2024-09-22T04:06:12.949322+0000","last_fullsized":"2024-09-22T04:06:12.949322+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.948895+0000","last_change":"2024-09-22T03:52:13.464870+0000","last_active":"2024-09-22T04:06:12.948895+0000","last_peered":"2024-09-22T04:06:12.948895+0000","last_clean":"2024-09-22T04:06:12.948895+0000","last_became_active":"2024-09-22T03:52:13.412673+0000","last_became_peered":"2024-09-22T03:52:13.412673+0000","last_unstale":"2024-09-22T04:06:12.948895+0000","last_undegraded":"2024-09-22T04:06:12.948895+0000","last_fullsized":"2024-09-22T04:06:12.948895+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.952801+0000","last_change":"2024-09-22T03:52:13.477357+0000","last_active":"2024-09-22T04:06:12.952801+0000","last_peered":"2024-09-22T04:06:12.952801+0000","last_clean":"2024-09-22T04:06:12.952801+0000","last_became_active":"2024-09-22T03:52:13.441995+0000","last_became_peered":"2024-09-22T03:52:13.441995+0000","last_unstale":"2024-09-22T04:06:12.952801+0000","last_undegraded":"2024-09-22T04:06:12.952801+0000","last_fullsized":"2024-09-22T04:06:12.952801+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.942831+0000","last_change":"2024-09-22T03:52:13.667385+0000","last_active":"2024-09-22T04:06:12.942831+0000","last_peered":"2024-09-22T04:06:12.942831+0000","last_clean":"2024-09-22T04:06:12.942831+0000","last_became_active":"2024-09-22T03:52:13.657594+0000","last_became_peered":"2024-09-22T03:52:13.657594+0000","last_unstale":"2024-09-22T04:06:12.942831+0000","last_undegraded":"2024-09-22T04:06:12.942831+0000","last_fullsized":"2024-09-22T04:06:12.942831+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.957227+0000","last_change":"2024-09-22T03:52:13.488158+0000","last_active":"2024-09-22T04:06:12.957227+0000","last_peered":"2024-09-22T04:06:12.957227+0000","last_clean":"2024-09-22T04:06:12.957227+0000","last_became_active":"2024-09-22T03:52:13.435529+0000","last_became_peered":"2024-09-22T03:52:13.435529+0000","last_unstale":"2024-09-22T04:06:12.957227+0000","last_undegraded":"2024-09-22T04:06:12.957227+0000","last_fullsized":"2024-09-22T04:06:12.957227+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.942075+0000","last_change":"2024-09-22T03:52:13.418644+0000","last_active":"2024-09-22T04:06:12.942075+0000","last_peered":"2024-09-22T04:06:12.942075+0000","last_clean":"2024-09-22T04:06:12.942075+0000","last_became_active":"2024-09-22T03:52:13.411413+0000","last_became_peered":"2024-09-22T03:52:13.411413+0000","last_unstale":"2024-09-22T04:06:12.942075+0000","last_undegraded":"2024-09-22T04:06:12.942075+0000","last_fullsized":"2024-09-22T04:06:12.942075+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.937201+0000","last_change":"2024-09-22T03:52:13.441353+0000","last_active":"2024-09-22T04:06:12.937201+0000","last_peered":"2024-09-22T04:06:12.937201+0000","last_clean":"2024-09-22T04:06:12.937201+0000","last_became_active":"2024-09-22T03:52:13.399486+0000","last_became_peered":"2024-09-22T03:52:13.399486+0000","last_unstale":"2024-09-22T04:06:12.937201+0000","last_undegraded":"2024-09-22T04:06:12.937201+0000","last_fullsized":"2024-09-22T04:06:12.937201+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.932480+0000","last_change":"2024-09-22T03:52:13.359382+0000","last_active":"2024-09-22T04:06:12.932480+0000","last_peered":"2024-09-22T04:06:12.932480+0000","last_clean":"2024-09-22T04:06:12.932480+0000","last_became_active":"2024-09-22T03:52:13.335952+0000","last_became_peered":"2024-09-22T03:52:13.335952+0000","last_unstale":"2024-09-22T04:06:12.932480+0000","last_undegraded":"2024-09-22T04:06:12.932480+0000","last_fullsized":"2024-09-22T04:06:12.932480+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.951565+0000","last_change":"2024-09-22T03:52:13.445595+0000","last_active":"2024-09-22T04:06:12.951565+0000","last_peered":"2024-09-22T04:06:12.951565+0000","last_clean":"2024-09-22T04:06:12.951565+0000","last_became_active":"2024-09-22T03:52:13.414407+0000","last_became_peered":"2024-09-22T03:52:13.414407+0000","last_unstale":"2024-09-22T04:06:12.951565+0000","last_undegraded":"2024-09-22T04:06:12.951565+0000","last_fullsized":"2024-09-22T04:06:12.951565+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.941331+0000","last_change":"2024-09-22T03:52:13.491741+0000","last_active":"2024-09-22T04:06:12.941331+0000","last_peered":"2024-09-22T04:06:12.941331+0000","last_clean":"2024-09-22T04:06:12.941331+0000","last_became_active":"2024-09-22T03:52:13.431192+0000","last_became_peered":"2024-09-22T03:52:13.431192+0000","last_unstale":"2024-09-22T04:06:12.941331+0000","last_undegraded":"2024-09-22T04:06:12.941331+0000","last_fullsized":"2024-09-22T04:06:12.941331+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.935509+0000","last_change":"2024-09-22T03:52:13.501197+0000","last_active":"2024-09-22T04:06:12.935509+0000","last_peered":"2024-09-22T04:06:12.935509+0000","last_clean":"2024-09-22T04:06:12.935509+0000","last_became_active":"2024-09-22T03:52:13.393363+0000","last_became_peered":"2024-09-22T03:52:13.393363+0000","last_unstale":"2024-09-22T04:06:12.935509+0000","last_undegraded":"2024-09-22T04:06:12.935509+0000","last_fullsized":"2024-09-22T04:06:12.935509+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.948186+0000","last_change":"2024-09-22T03:52:13.454940+0000","last_active":"2024-09-22T04:06:12.948186+0000","last_peered":"2024-09-22T04:06:12.948186+0000","last_clean":"2024-09-22T04:06:12.948186+0000","last_became_active":"2024-09-22T03:52:13.423424+0000","last_became_peered":"2024-09-22T03:52:13.423424+0000","last_unstale":"2024-09-22T04:06:12.948186+0000","last_undegraded":"2024-09-22T04:06:12.948186+0000","last_fullsized":"2024-09-22T04:06:12.948186+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.943291+0000","last_change":"2024-09-22T03:52:13.491666+0000","last_active":"2024-09-22T04:06:12.943291+0000","last_peered":"2024-09-22T04:06:12.943291+0000","last_clean":"2024-09-22T04:06:12.943291+0000","last_became_active":"2024-09-22T03:52:13.404954+0000","last_became_peered":"2024-09-22T03:52:13.404954+0000","last_unstale":"2024-09-22T04:06:12.943291+0000","last_undegraded":"2024-09-22T04:06:12.943291+0000","last_fullsized":"2024-09-22T04:06:12.943291+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"241'7","reported_seq":1491,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.961124+0000","last_change":"2024-09-22T03:52:13.469238+0000","last_active":"2024-09-22T04:06:12.961124+0000","last_peered":"2024-09-22T04:06:12.961124+0000","last_clean":"2024-09-22T04:06:12.961124+0000","last_became_active":"2024-09-22T03:52:13.422437+0000","last_became_peered":"2024-09-22T03:52:13.422437+0000","last_unstale":"2024-09-22T04:06:12.961124+0000","last_undegraded":"2024-09-22T04:06:12.961124+0000","last_fullsized":"2024-09-22T04:06:12.961124+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.949497+0000","last_change":"2024-09-22T03:52:13.460700+0000","last_active":"2024-09-22T04:06:12.949497+0000","last_peered":"2024-09-22T04:06:12.949497+0000","last_clean":"2024-09-22T04:06:12.949497+0000","last_became_active":"2024-09-22T03:52:13.420437+0000","last_became_peered":"2024-09-22T03:52:13.420437+0000","last_unstale":"2024-09-22T04:06:12.949497+0000","last_undegraded":"2024-09-22T04:06:12.949497+0000","last_fullsized":"2024-09-22T04:06:12.949497+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":2163,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.953799+0000","last_change":"2024-09-22T03:46:57.335250+0000","last_active":"2024-09-22T04:06:12.953799+0000","last_peered":"2024-09-22T04:06:12.953799+0000","last_clean":"2024-09-22T04:06:12.953799+0000","last_became_active":"2024-09-22T03:46:55.378262+0000","last_became_peered":"2024-09-22T03:46:55.378262+0000","last_unstale":"2024-09-22T04:06:12.953799+0000","last_undegraded":"2024-09-22T04:06:12.953799+0000","last_fullsized":"2024-09-22T04:06:12.953799+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2163,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.943814+0000","last_change":"2024-09-22T03:46:57.333626+0000","last_active":"2024-09-22T04:06:12.943814+0000","last_peered":"2024-09-22T04:06:12.943814+0000","last_clean":"2024-09-22T04:06:12.943814+0000","last_became_active":"2024-09-22T03:46:55.378604+0000","last_became_peered":"2024-09-22T03:46:55.378604+0000","last_unstale":"2024-09-22T04:06:12.943814+0000","last_undegraded":"2024-09-22T04:06:12.943814+0000","last_fullsized":"2024-09-22T04:06:12.943814+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2163,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.951973+0000","last_change":"2024-09-22T03:46:57.329704+0000","last_active":"2024-09-22T04:06:12.951973+0000","last_peered":"2024-09-22T04:06:12.951973+0000","last_clean":"2024-09-22T04:06:12.951973+0000","last_became_active":"2024-09-22T03:46:55.370620+0000","last_became_peered":"2024-09-22T03:46:55.370620+0000","last_unstale":"2024-09-22T04:06:12.951973+0000","last_undegraded":"2024-09-22T04:06:12.951973+0000","last_fullsized":"2024-09-22T04:06:12.951973+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2163,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.935759+0000","last_change":"2024-09-22T03:46:57.328548+0000","last_active":"2024-09-22T04:06:12.935759+0000","last_peered":"2024-09-22T04:06:12.935759+0000","last_clean":"2024-09-22T04:06:12.935759+0000","last_became_active":"2024-09-22T03:46:55.358980+0000","last_became_peered":"2024-09-22T03:46:55.358980+0000","last_unstale":"2024-09-22T04:06:12.935759+0000","last_undegraded":"2024-09-22T04:06:12.935759+0000","last_fullsized":"2024-09-22T04:06:12.935759+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":2163,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.949682+0000","last_change":"2024-09-22T03:46:57.331270+0000","last_active":"2024-09-22T04:06:12.949682+0000","last_peered":"2024-09-22T04:06:12.949682+0000","last_clean":"2024-09-22T04:06:12.949682+0000","last_became_active":"2024-09-22T03:46:55.383021+0000","last_became_peered":"2024-09-22T03:46:55.383021+0000","last_unstale":"2024-09-22T04:06:12.949682+0000","last_undegraded":"2024-09-22T04:06:12.949682+0000","last_fullsized":"2024-09-22T04:06:12.949682+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2163,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.943350+0000","last_change":"2024-09-22T03:46:57.329346+0000","last_active":"2024-09-22T04:06:12.943350+0000","last_peered":"2024-09-22T04:06:12.943350+0000","last_clean":"2024-09-22T04:06:12.943350+0000","last_became_active":"2024-09-22T03:46:55.358542+0000","last_became_peered":"2024-09-22T03:46:55.358542+0000","last_unstale":"2024-09-22T04:06:12.943350+0000","last_undegraded":"2024-09-22T04:06:12.943350+0000","last_fullsized":"2024-09-22T04:06:12.943350+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2161,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.943416+0000","last_change":"2024-09-22T03:46:57.332056+0000","last_active":"2024-09-22T04:06:12.943416+0000","last_peered":"2024-09-22T04:06:12.943416+0000","last_clean":"2024-09-22T04:06:12.943416+0000","last_became_active":"2024-09-22T03:46:55.412604+0000","last_became_peered":"2024-09-22T03:46:55.412604+0000","last_unstale":"2024-09-22T04:06:12.943416+0000","last_undegraded":"2024-09-22T04:06:12.943416+0000","last_fullsized":"2024-09-22T04:06:12.943416+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":2164,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.958818+0000","last_change":"2024-09-22T03:46:57.335158+0000","last_active":"2024-09-22T04:06:12.958818+0000","last_peered":"2024-09-22T04:06:12.958818+0000","last_clean":"2024-09-22T04:06:12.958818+0000","last_became_active":"2024-09-22T03:46:55.387128+0000","last_became_peered":"2024-09-22T03:46:55.387128+0000","last_unstale":"2024-09-22T04:06:12.958818+0000","last_undegraded":"2024-09-22T04:06:12.958818+0000","last_fullsized":"2024-09-22T04:06:12.958818+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":2235,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.936476+0000","last_change":"2024-09-22T03:46:53.370744+0000","last_active":"2024-09-22T04:06:12.936476+0000","last_peered":"2024-09-22T04:06:12.936476+0000","last_clean":"2024-09-22T04:06:12.936476+0000","last_became_active":"2024-09-22T03:46:53.349788+0000","last_became_peered":"2024-09-22T03:46:53.349788+0000","last_unstale":"2024-09-22T04:06:12.936476+0000","last_undegraded":"2024-09-22T04:06:12.936476+0000","last_fullsized":"2024-09-22T04:06:12.936476+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:52.275666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:52.275666+0000","last_clean_scrub_stamp":"2024-09-22T03:46:52.275666+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214187458560,"internally_reserved":0,"allocated":24035328,"data_stored":24035328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214187458560,"internally_reserved":0,"allocated":24035328,"data_stored":24035328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53546475520,"internally_reserved":0,"allocated":6397952,"data_stored":6397952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":6248,"kb_used_data":6248,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52291480,"statfs":{"total":53552873472,"available":53546475520,"internally_reserved":0,"allocated":6397952,"data_stored":6397952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":5300,"kb_used_data":5300,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52292428,"statfs":{"total":53552873472,"available":53547446272,"internally_reserved":0,"allocated":5427200,"data_stored":5427200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":5128,"kb_used_data":5128,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52292600,"statfs":{"total":53552873472,"available":53547622400,"internally_reserved":0,"allocated":5251072,"data_stored":5251072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":6796,"kb_used_data":6796,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52290932,"statfs":{"total":53552873472,"available":53545914368,"internally_reserved":0,"allocated":6959104,"data_stored":6959104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53546475520,"internally_reserved":0,"allocated":6397952,"data_stored":6397952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53545914368,"internally_reserved":0,"allocated":6959104,"data_stored":6959104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53547622400,"internally_reserved":0,"allocated":5251072,"data_stored":5251072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53547446272,"internally_reserved":0,"allocated":5427200,"data_stored":5427200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53546475520,"internally_reserved":0,"allocated":6397952,"data_stored":6397952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":0,"total":53552873472,"available":53545914368,"internally_reserved":0,"allocated":6959104,"data_stored":6959104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":1,"total":53552873472,"available":53547622400,"internally_reserved":0,"allocated":5251072,"data_stored":5251072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":2,"total":53552873472,"available":53547446272,"internally_reserved":0,"allocated":5427200,"data_stored":5427200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":3,"total":53552873472,"available":53546475520,"internally_reserved":0,"allocated":6397952,"data_stored":6397952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T04:06:45.495 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T04:06:45.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.620+0000 7fa7bb591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:45.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.625+0000 7fa7bb591640 1 Processor -- start 2024-09-22T04:06:45.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.625+0000 7fa7bb591640 1 -- start start 2024-09-22T04:06:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.627+0000 7fa7bb591640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa7b40a3e10 0x7fa7b40a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.627+0000 7fa7bb591640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b406ec50 con 0x7fa7b40a4760 2024-09-22T04:06:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.627+0000 7fa7bb591640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa7b40a6130 con 0x7fa7b40a3e10 2024-09-22T04:06:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.627+0000 7fa7b932d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa7b40a3e10 0x7fa7b40a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.627+0000 7fa7b932d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa7b40a3e10 0x7fa7b40a4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34434/0 (socket says 172.21.15.3:34434) 2024-09-22T04:06:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.627+0000 7fa7b8b2c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa7b40a4760 0x7fa7b4135e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45280/0 (socket says 172.21.15.3:45280) 2024-09-22T04:06:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.627+0000 7fa7b932d640 1 -- 172.21.15.3:0/2988075104 learned_addr learned my addr 172.21.15.3:0/2988075104 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.627+0000 7fa7b932d640 1 -- 172.21.15.3:0/2988075104 >> v1:172.21.15.3:6789/0 conn(0x7fa7b40a4760 legacy=0x7fa7b4135e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.627+0000 7fa7b932d640 1 -- 172.21.15.3:0/2988075104 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b40a5110 con 0x7fa7b40a3e10 2024-09-22T04:06:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.628+0000 7fa7b932d640 1 --2- 172.21.15.3:0/2988075104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7b40a3e10 0x7fa7b40a4220 crc :-1 s=READY pgs=13006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91f1a5fed7f8ad4 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.628+0000 7fa7abfff640 1 -- 172.21.15.3:0/2988075104 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa7b0009e50 con 0x7fa7b40a3e10 2024-09-22T04:06:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.628+0000 7fa7abfff640 1 -- 172.21.15.3:0/2988075104 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa7b000e230 con 0x7fa7b40a3e10 2024-09-22T04:06:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.628+0000 7fa7abfff640 1 -- 172.21.15.3:0/2988075104 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa7b000e4d0 con 0x7fa7b40a3e10 2024-09-22T04:06:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.629+0000 7fa7bb591640 1 -- 172.21.15.3:0/2988075104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7b40a3e10 msgr2=0x7fa7b40a4220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.629+0000 7fa7bb591640 1 --2- 172.21.15.3:0/2988075104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7b40a3e10 0x7fa7b40a4220 crc :-1 s=READY pgs=13006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.629+0000 7fa7bb591640 1 -- 172.21.15.3:0/2988075104 shutdown_connections 2024-09-22T04:06:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.629+0000 7fa7bb591640 1 --2- 172.21.15.3:0/2988075104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7b40a3e10 0x7fa7b40a4220 unknown :-1 s=CLOSED pgs=13006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.629+0000 7fa7bb591640 1 -- 172.21.15.3:0/2988075104 >> 172.21.15.3:0/2988075104 conn(0x7fa7b40a74f0 msgr2=0x7fa7b40a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.629+0000 7fa7bb591640 1 -- 172.21.15.3:0/2988075104 shutdown_connections 2024-09-22T04:06:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.629+0000 7fa7bb591640 1 -- 172.21.15.3:0/2988075104 wait complete. 2024-09-22T04:06:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.630+0000 7fa7bb591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.632+0000 7fa7bb591640 1 Processor -- start 2024-09-22T04:06:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.632+0000 7fa7bb591640 1 -- start start 2024-09-22T04:06:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.633+0000 7fa7bb591640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7b40a3e10 0x7fa7b4123be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:45.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.633+0000 7fa7bb591640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b41278e0 con 0x7fa7b40a3e10 2024-09-22T04:06:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.634+0000 7fa7b932d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7b40a3e10 0x7fa7b4123be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.634+0000 7fa7b932d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7b40a3e10 0x7fa7b4123be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34440/0 (socket says 172.21.15.3:34440) 2024-09-22T04:06:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.634+0000 7fa7b932d640 1 -- 172.21.15.3:0/444725153 learned_addr learned my addr 172.21.15.3:0/444725153 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.634+0000 7fa7b932d640 1 -- 172.21.15.3:0/444725153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b4125620 con 0x7fa7b40a3e10 2024-09-22T04:06:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.634+0000 7fa7b932d640 1 --2- 172.21.15.3:0/444725153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7b40a3e10 0x7fa7b4123be0 crc :-1 s=READY pgs=13007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.634+0000 7fa7a9ffb640 1 -- 172.21.15.3:0/444725153 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa7b000eda0 con 0x7fa7b40a3e10 2024-09-22T04:06:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.634+0000 7fa7a9ffb640 1 -- 172.21.15.3:0/444725153 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa7b0015420 con 0x7fa7b40a3e10 2024-09-22T04:06:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.634+0000 7fa7a9ffb640 1 -- 172.21.15.3:0/444725153 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa7b00156c0 con 0x7fa7b40a3e10 2024-09-22T04:06:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.634+0000 7fa7bb591640 1 -- 172.21.15.3:0/444725153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b4124d70 con 0x7fa7b40a3e10 2024-09-22T04:06:45.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.635+0000 7fa7bb591640 1 -- 172.21.15.3:0/444725153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b4126f00 con 0x7fa7b40a3e10 2024-09-22T04:06:45.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.636+0000 7fa7bb591640 1 -- 172.21.15.3:0/444725153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b40a56d0 con 0x7fa7b40a3e10 2024-09-22T04:06:45.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.636+0000 7fa7a9ffb640 1 -- 172.21.15.3:0/444725153 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fa7b0015870 con 0x7fa7b40a3e10 2024-09-22T04:06:45.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.636+0000 7fa7a9ffb640 1 --2- 172.21.15.3:0/444725153 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa79803ead0 0x7fa798040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:45.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.636+0000 7fa7a9ffb640 1 -- 172.21.15.3:0/444725153 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(733..733 src has 590..733) ==== 3431+0+0 (crc 0 0 0) 0x7fa7b0051180 con 0x7fa7b40a3e10 2024-09-22T04:06:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.636+0000 7fa7b8b2c640 1 --2- 172.21.15.3:0/444725153 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa79803ead0 0x7fa798040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.637+0000 7fa7b8b2c640 1 --2- 172.21.15.3:0/444725153 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa79803ead0 0x7fa798040fe0 crc :-1 s=READY pgs=3870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:45.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.641+0000 7fa7a9ffb640 1 -- 172.21.15.3:0/444725153 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7fa7b0014c60 con 0x7fa7b40a3e10 2024-09-22T04:06:45.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.830+0000 7fa7bb591640 1 -- 172.21.15.3:0/444725153 --> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa7b40737f0 con 0x7fa79803ead0 2024-09-22T04:06:45.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.833+0000 7fa7a9ffb640 1 -- 172.21.15.3:0/444725153 <== mgr.4099 v2:172.21.15.123:6800/551784226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+109430 (crc 0 0 0) 0x7fa7b40737f0 con 0x7fa79803ead0 2024-09-22T04:06:45.835 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T04:06:45.835 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-22T04:06:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.835+0000 7fa7bb591640 1 -- 172.21.15.3:0/444725153 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa79803ead0 msgr2=0x7fa798040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.835+0000 7fa7bb591640 1 --2- 172.21.15.3:0/444725153 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa79803ead0 0x7fa798040fe0 crc :-1 s=READY pgs=3870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.835+0000 7fa7bb591640 1 -- 172.21.15.3:0/444725153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7b40a3e10 msgr2=0x7fa7b4123be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.835+0000 7fa7bb591640 1 --2- 172.21.15.3:0/444725153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7b40a3e10 0x7fa7b4123be0 crc :-1 s=READY pgs=13007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.835+0000 7fa7bb591640 1 -- 172.21.15.3:0/444725153 shutdown_connections 2024-09-22T04:06:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.835+0000 7fa7bb591640 1 --2- 172.21.15.3:0/444725153 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa79803ead0 0x7fa798040fe0 unknown :-1 s=CLOSED pgs=3870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.835+0000 7fa7bb591640 1 --2- 172.21.15.3:0/444725153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7b40a3e10 0x7fa7b4123be0 unknown :-1 s=CLOSED pgs=13007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.835+0000 7fa7bb591640 1 -- 172.21.15.3:0/444725153 >> 172.21.15.3:0/444725153 conn(0x7fa7b40a74f0 msgr2=0x7fa7b41bbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.835+0000 7fa7bb591640 1 -- 172.21.15.3:0/444725153 shutdown_connections 2024-09-22T04:06:45.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.835+0000 7fa7bb591640 1 -- 172.21.15.3:0/444725153 wait complete. 2024-09-22T04:06:45.848 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":1324,"stamp":"2024-09-22T04:06:44.424778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590392,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":52,"num_write":88,"num_write_kb":1115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62,"ondisk_log_size":62,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":23472,"kb_used_data":23472,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209167440,"statfs":{"total":214211493888,"available":214187458560,"internally_reserved":0,"allocated":24035328,"data_stored":24035328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-321875,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-25,"num_write_kb":-325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.585809"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.941328+0000","last_change":"2024-09-22T03:52:13.461984+0000","last_active":"2024-09-22T04:06:12.941328+0000","last_peered":"2024-09-22T04:06:12.941328+0000","last_clean":"2024-09-22T04:06:12.941328+0000","last_became_active":"2024-09-22T03:52:13.436104+0000","last_became_peered":"2024-09-22T03:52:13.436104+0000","last_unstale":"2024-09-22T04:06:12.941328+0000","last_undegraded":"2024-09-22T04:06:12.941328+0000","last_fullsized":"2024-09-22T04:06:12.941328+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.950121+0000","last_change":"2024-09-22T03:52:13.517533+0000","last_active":"2024-09-22T04:06:12.950121+0000","last_peered":"2024-09-22T04:06:12.950121+0000","last_clean":"2024-09-22T04:06:12.950121+0000","last_became_active":"2024-09-22T03:52:13.485941+0000","last_became_peered":"2024-09-22T03:52:13.485941+0000","last_unstale":"2024-09-22T04:06:12.950121+0000","last_undegraded":"2024-09-22T04:06:12.950121+0000","last_fullsized":"2024-09-22T04:06:12.950121+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.939878+0000","last_change":"2024-09-22T03:52:13.441983+0000","last_active":"2024-09-22T04:06:12.939878+0000","last_peered":"2024-09-22T04:06:12.939878+0000","last_clean":"2024-09-22T04:06:12.939878+0000","last_became_active":"2024-09-22T03:52:13.435054+0000","last_became_peered":"2024-09-22T03:52:13.435054+0000","last_unstale":"2024-09-22T04:06:12.939878+0000","last_undegraded":"2024-09-22T04:06:12.939878+0000","last_fullsized":"2024-09-22T04:06:12.939878+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.938208+0000","last_change":"2024-09-22T03:52:13.475862+0000","last_active":"2024-09-22T04:06:12.938208+0000","last_peered":"2024-09-22T04:06:12.938208+0000","last_clean":"2024-09-22T04:06:12.938208+0000","last_became_active":"2024-09-22T03:52:13.424125+0000","last_became_peered":"2024-09-22T03:52:13.424125+0000","last_unstale":"2024-09-22T04:06:12.938208+0000","last_undegraded":"2024-09-22T04:06:12.938208+0000","last_fullsized":"2024-09-22T04:06:12.938208+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.959684+0000","last_change":"2024-09-22T03:52:13.638357+0000","last_active":"2024-09-22T04:06:12.959684+0000","last_peered":"2024-09-22T04:06:12.959684+0000","last_clean":"2024-09-22T04:06:12.959684+0000","last_became_active":"2024-09-22T03:52:13.612276+0000","last_became_peered":"2024-09-22T03:52:13.612276+0000","last_unstale":"2024-09-22T04:06:12.959684+0000","last_undegraded":"2024-09-22T04:06:12.959684+0000","last_fullsized":"2024-09-22T04:06:12.959684+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.955663+0000","last_change":"2024-09-22T03:52:13.631942+0000","last_active":"2024-09-22T04:06:12.955663+0000","last_peered":"2024-09-22T04:06:12.955663+0000","last_clean":"2024-09-22T04:06:12.955663+0000","last_became_active":"2024-09-22T03:52:13.584449+0000","last_became_peered":"2024-09-22T03:52:13.584449+0000","last_unstale":"2024-09-22T04:06:12.955663+0000","last_undegraded":"2024-09-22T04:06:12.955663+0000","last_fullsized":"2024-09-22T04:06:12.955663+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.943719+0000","last_change":"2024-09-22T03:52:13.462393+0000","last_active":"2024-09-22T04:06:12.943719+0000","last_peered":"2024-09-22T04:06:12.943719+0000","last_clean":"2024-09-22T04:06:12.943719+0000","last_became_active":"2024-09-22T03:52:13.438920+0000","last_became_peered":"2024-09-22T03:52:13.438920+0000","last_unstale":"2024-09-22T04:06:12.943719+0000","last_undegraded":"2024-09-22T04:06:12.943719+0000","last_fullsized":"2024-09-22T04:06:12.943719+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.955610+0000","last_change":"2024-09-22T03:52:13.435067+0000","last_active":"2024-09-22T04:06:12.955610+0000","last_peered":"2024-09-22T04:06:12.955610+0000","last_clean":"2024-09-22T04:06:12.955610+0000","last_became_active":"2024-09-22T03:52:13.424399+0000","last_became_peered":"2024-09-22T03:52:13.424399+0000","last_unstale":"2024-09-22T04:06:12.955610+0000","last_undegraded":"2024-09-22T04:06:12.955610+0000","last_fullsized":"2024-09-22T04:06:12.955610+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.935572+0000","last_change":"2024-09-22T03:52:13.414909+0000","last_active":"2024-09-22T04:06:12.935572+0000","last_peered":"2024-09-22T04:06:12.935572+0000","last_clean":"2024-09-22T04:06:12.935572+0000","last_became_active":"2024-09-22T03:52:13.367950+0000","last_became_peered":"2024-09-22T03:52:13.367950+0000","last_unstale":"2024-09-22T04:06:12.935572+0000","last_undegraded":"2024-09-22T04:06:12.935572+0000","last_fullsized":"2024-09-22T04:06:12.935572+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.936631+0000","last_change":"2024-09-22T03:52:13.421431+0000","last_active":"2024-09-22T04:06:12.936631+0000","last_peered":"2024-09-22T04:06:12.936631+0000","last_clean":"2024-09-22T04:06:12.936631+0000","last_became_active":"2024-09-22T03:52:13.370886+0000","last_became_peered":"2024-09-22T03:52:13.370886+0000","last_unstale":"2024-09-22T04:06:12.936631+0000","last_undegraded":"2024-09-22T04:06:12.936631+0000","last_fullsized":"2024-09-22T04:06:12.936631+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.946060+0000","last_change":"2024-09-22T03:52:13.488521+0000","last_active":"2024-09-22T04:06:12.946060+0000","last_peered":"2024-09-22T04:06:12.946060+0000","last_clean":"2024-09-22T04:06:12.946060+0000","last_became_active":"2024-09-22T03:52:13.477618+0000","last_became_peered":"2024-09-22T03:52:13.477618+0000","last_unstale":"2024-09-22T04:06:12.946060+0000","last_undegraded":"2024-09-22T04:06:12.946060+0000","last_fullsized":"2024-09-22T04:06:12.946060+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.942381+0000","last_change":"2024-09-22T03:52:13.423108+0000","last_active":"2024-09-22T04:06:12.942381+0000","last_peered":"2024-09-22T04:06:12.942381+0000","last_clean":"2024-09-22T04:06:12.942381+0000","last_became_active":"2024-09-22T03:52:13.354826+0000","last_became_peered":"2024-09-22T03:52:13.354826+0000","last_unstale":"2024-09-22T04:06:12.942381+0000","last_undegraded":"2024-09-22T04:06:12.942381+0000","last_fullsized":"2024-09-22T04:06:12.942381+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.939191+0000","last_change":"2024-09-22T03:52:13.456498+0000","last_active":"2024-09-22T04:06:12.939191+0000","last_peered":"2024-09-22T04:06:12.939191+0000","last_clean":"2024-09-22T04:06:12.939191+0000","last_became_active":"2024-09-22T03:52:13.405184+0000","last_became_peered":"2024-09-22T03:52:13.405184+0000","last_unstale":"2024-09-22T04:06:12.939191+0000","last_undegraded":"2024-09-22T04:06:12.939191+0000","last_fullsized":"2024-09-22T04:06:12.939191+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.943494+0000","last_change":"2024-09-22T03:52:13.548435+0000","last_active":"2024-09-22T04:06:12.943494+0000","last_peered":"2024-09-22T04:06:12.943494+0000","last_clean":"2024-09-22T04:06:12.943494+0000","last_became_active":"2024-09-22T03:52:13.533286+0000","last_became_peered":"2024-09-22T03:52:13.533286+0000","last_unstale":"2024-09-22T04:06:12.943494+0000","last_undegraded":"2024-09-22T04:06:12.943494+0000","last_fullsized":"2024-09-22T04:06:12.943494+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.944076+0000","last_change":"2024-09-22T03:52:13.555314+0000","last_active":"2024-09-22T04:06:12.944076+0000","last_peered":"2024-09-22T04:06:12.944076+0000","last_clean":"2024-09-22T04:06:12.944076+0000","last_became_active":"2024-09-22T03:52:13.531887+0000","last_became_peered":"2024-09-22T03:52:13.531887+0000","last_unstale":"2024-09-22T04:06:12.944076+0000","last_undegraded":"2024-09-22T04:06:12.944076+0000","last_fullsized":"2024-09-22T04:06:12.944076+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.942361+0000","last_change":"2024-09-22T03:52:13.576998+0000","last_active":"2024-09-22T04:06:12.942361+0000","last_peered":"2024-09-22T04:06:12.942361+0000","last_clean":"2024-09-22T04:06:12.942361+0000","last_became_active":"2024-09-22T03:52:13.562231+0000","last_became_peered":"2024-09-22T03:52:13.562231+0000","last_unstale":"2024-09-22T04:06:12.942361+0000","last_undegraded":"2024-09-22T04:06:12.942361+0000","last_fullsized":"2024-09-22T04:06:12.942361+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.952801+0000","last_change":"2024-09-22T03:52:13.527047+0000","last_active":"2024-09-22T04:06:12.952801+0000","last_peered":"2024-09-22T04:06:12.952801+0000","last_clean":"2024-09-22T04:06:12.952801+0000","last_became_active":"2024-09-22T03:52:13.496766+0000","last_became_peered":"2024-09-22T03:52:13.496766+0000","last_unstale":"2024-09-22T04:06:12.952801+0000","last_undegraded":"2024-09-22T04:06:12.952801+0000","last_fullsized":"2024-09-22T04:06:12.952801+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.949322+0000","last_change":"2024-09-22T03:52:13.517430+0000","last_active":"2024-09-22T04:06:12.949322+0000","last_peered":"2024-09-22T04:06:12.949322+0000","last_clean":"2024-09-22T04:06:12.949322+0000","last_became_active":"2024-09-22T03:52:13.496886+0000","last_became_peered":"2024-09-22T03:52:13.496886+0000","last_unstale":"2024-09-22T04:06:12.949322+0000","last_undegraded":"2024-09-22T04:06:12.949322+0000","last_fullsized":"2024-09-22T04:06:12.949322+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.948895+0000","last_change":"2024-09-22T03:52:13.464870+0000","last_active":"2024-09-22T04:06:12.948895+0000","last_peered":"2024-09-22T04:06:12.948895+0000","last_clean":"2024-09-22T04:06:12.948895+0000","last_became_active":"2024-09-22T03:52:13.412673+0000","last_became_peered":"2024-09-22T03:52:13.412673+0000","last_unstale":"2024-09-22T04:06:12.948895+0000","last_undegraded":"2024-09-22T04:06:12.948895+0000","last_fullsized":"2024-09-22T04:06:12.948895+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.952801+0000","last_change":"2024-09-22T03:52:13.477357+0000","last_active":"2024-09-22T04:06:12.952801+0000","last_peered":"2024-09-22T04:06:12.952801+0000","last_clean":"2024-09-22T04:06:12.952801+0000","last_became_active":"2024-09-22T03:52:13.441995+0000","last_became_peered":"2024-09-22T03:52:13.441995+0000","last_unstale":"2024-09-22T04:06:12.952801+0000","last_undegraded":"2024-09-22T04:06:12.952801+0000","last_fullsized":"2024-09-22T04:06:12.952801+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.942831+0000","last_change":"2024-09-22T03:52:13.667385+0000","last_active":"2024-09-22T04:06:12.942831+0000","last_peered":"2024-09-22T04:06:12.942831+0000","last_clean":"2024-09-22T04:06:12.942831+0000","last_became_active":"2024-09-22T03:52:13.657594+0000","last_became_peered":"2024-09-22T03:52:13.657594+0000","last_unstale":"2024-09-22T04:06:12.942831+0000","last_undegraded":"2024-09-22T04:06:12.942831+0000","last_fullsized":"2024-09-22T04:06:12.942831+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.957227+0000","last_change":"2024-09-22T03:52:13.488158+0000","last_active":"2024-09-22T04:06:12.957227+0000","last_peered":"2024-09-22T04:06:12.957227+0000","last_clean":"2024-09-22T04:06:12.957227+0000","last_became_active":"2024-09-22T03:52:13.435529+0000","last_became_peered":"2024-09-22T03:52:13.435529+0000","last_unstale":"2024-09-22T04:06:12.957227+0000","last_undegraded":"2024-09-22T04:06:12.957227+0000","last_fullsized":"2024-09-22T04:06:12.957227+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.942075+0000","last_change":"2024-09-22T03:52:13.418644+0000","last_active":"2024-09-22T04:06:12.942075+0000","last_peered":"2024-09-22T04:06:12.942075+0000","last_clean":"2024-09-22T04:06:12.942075+0000","last_became_active":"2024-09-22T03:52:13.411413+0000","last_became_peered":"2024-09-22T03:52:13.411413+0000","last_unstale":"2024-09-22T04:06:12.942075+0000","last_undegraded":"2024-09-22T04:06:12.942075+0000","last_fullsized":"2024-09-22T04:06:12.942075+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.937201+0000","last_change":"2024-09-22T03:52:13.441353+0000","last_active":"2024-09-22T04:06:12.937201+0000","last_peered":"2024-09-22T04:06:12.937201+0000","last_clean":"2024-09-22T04:06:12.937201+0000","last_became_active":"2024-09-22T03:52:13.399486+0000","last_became_peered":"2024-09-22T03:52:13.399486+0000","last_unstale":"2024-09-22T04:06:12.937201+0000","last_undegraded":"2024-09-22T04:06:12.937201+0000","last_fullsized":"2024-09-22T04:06:12.937201+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.932480+0000","last_change":"2024-09-22T03:52:13.359382+0000","last_active":"2024-09-22T04:06:12.932480+0000","last_peered":"2024-09-22T04:06:12.932480+0000","last_clean":"2024-09-22T04:06:12.932480+0000","last_became_active":"2024-09-22T03:52:13.335952+0000","last_became_peered":"2024-09-22T03:52:13.335952+0000","last_unstale":"2024-09-22T04:06:12.932480+0000","last_undegraded":"2024-09-22T04:06:12.932480+0000","last_fullsized":"2024-09-22T04:06:12.932480+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.951565+0000","last_change":"2024-09-22T03:52:13.445595+0000","last_active":"2024-09-22T04:06:12.951565+0000","last_peered":"2024-09-22T04:06:12.951565+0000","last_clean":"2024-09-22T04:06:12.951565+0000","last_became_active":"2024-09-22T03:52:13.414407+0000","last_became_peered":"2024-09-22T03:52:13.414407+0000","last_unstale":"2024-09-22T04:06:12.951565+0000","last_undegraded":"2024-09-22T04:06:12.951565+0000","last_fullsized":"2024-09-22T04:06:12.951565+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.941331+0000","last_change":"2024-09-22T03:52:13.491741+0000","last_active":"2024-09-22T04:06:12.941331+0000","last_peered":"2024-09-22T04:06:12.941331+0000","last_clean":"2024-09-22T04:06:12.941331+0000","last_became_active":"2024-09-22T03:52:13.431192+0000","last_became_peered":"2024-09-22T03:52:13.431192+0000","last_unstale":"2024-09-22T04:06:12.941331+0000","last_undegraded":"2024-09-22T04:06:12.941331+0000","last_fullsized":"2024-09-22T04:06:12.941331+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.935509+0000","last_change":"2024-09-22T03:52:13.501197+0000","last_active":"2024-09-22T04:06:12.935509+0000","last_peered":"2024-09-22T04:06:12.935509+0000","last_clean":"2024-09-22T04:06:12.935509+0000","last_became_active":"2024-09-22T03:52:13.393363+0000","last_became_peered":"2024-09-22T03:52:13.393363+0000","last_unstale":"2024-09-22T04:06:12.935509+0000","last_undegraded":"2024-09-22T04:06:12.935509+0000","last_fullsized":"2024-09-22T04:06:12.935509+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.948186+0000","last_change":"2024-09-22T03:52:13.454940+0000","last_active":"2024-09-22T04:06:12.948186+0000","last_peered":"2024-09-22T04:06:12.948186+0000","last_clean":"2024-09-22T04:06:12.948186+0000","last_became_active":"2024-09-22T03:52:13.423424+0000","last_became_peered":"2024-09-22T03:52:13.423424+0000","last_unstale":"2024-09-22T04:06:12.948186+0000","last_undegraded":"2024-09-22T04:06:12.948186+0000","last_fullsized":"2024-09-22T04:06:12.948186+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.943291+0000","last_change":"2024-09-22T03:52:13.491666+0000","last_active":"2024-09-22T04:06:12.943291+0000","last_peered":"2024-09-22T04:06:12.943291+0000","last_clean":"2024-09-22T04:06:12.943291+0000","last_became_active":"2024-09-22T03:52:13.404954+0000","last_became_peered":"2024-09-22T03:52:13.404954+0000","last_unstale":"2024-09-22T04:06:12.943291+0000","last_undegraded":"2024-09-22T04:06:12.943291+0000","last_fullsized":"2024-09-22T04:06:12.943291+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"241'7","reported_seq":1491,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.961124+0000","last_change":"2024-09-22T03:52:13.469238+0000","last_active":"2024-09-22T04:06:12.961124+0000","last_peered":"2024-09-22T04:06:12.961124+0000","last_clean":"2024-09-22T04:06:12.961124+0000","last_became_active":"2024-09-22T03:52:13.422437+0000","last_became_peered":"2024-09-22T03:52:13.422437+0000","last_unstale":"2024-09-22T04:06:12.961124+0000","last_undegraded":"2024-09-22T04:06:12.961124+0000","last_fullsized":"2024-09-22T04:06:12.961124+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1478,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.949497+0000","last_change":"2024-09-22T03:52:13.460700+0000","last_active":"2024-09-22T04:06:12.949497+0000","last_peered":"2024-09-22T04:06:12.949497+0000","last_clean":"2024-09-22T04:06:12.949497+0000","last_became_active":"2024-09-22T03:52:13.420437+0000","last_became_peered":"2024-09-22T03:52:13.420437+0000","last_unstale":"2024-09-22T04:06:12.949497+0000","last_undegraded":"2024-09-22T04:06:12.949497+0000","last_fullsized":"2024-09-22T04:06:12.949497+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":2163,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.953799+0000","last_change":"2024-09-22T03:46:57.335250+0000","last_active":"2024-09-22T04:06:12.953799+0000","last_peered":"2024-09-22T04:06:12.953799+0000","last_clean":"2024-09-22T04:06:12.953799+0000","last_became_active":"2024-09-22T03:46:55.378262+0000","last_became_peered":"2024-09-22T03:46:55.378262+0000","last_unstale":"2024-09-22T04:06:12.953799+0000","last_undegraded":"2024-09-22T04:06:12.953799+0000","last_fullsized":"2024-09-22T04:06:12.953799+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2163,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.943814+0000","last_change":"2024-09-22T03:46:57.333626+0000","last_active":"2024-09-22T04:06:12.943814+0000","last_peered":"2024-09-22T04:06:12.943814+0000","last_clean":"2024-09-22T04:06:12.943814+0000","last_became_active":"2024-09-22T03:46:55.378604+0000","last_became_peered":"2024-09-22T03:46:55.378604+0000","last_unstale":"2024-09-22T04:06:12.943814+0000","last_undegraded":"2024-09-22T04:06:12.943814+0000","last_fullsized":"2024-09-22T04:06:12.943814+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2163,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.951973+0000","last_change":"2024-09-22T03:46:57.329704+0000","last_active":"2024-09-22T04:06:12.951973+0000","last_peered":"2024-09-22T04:06:12.951973+0000","last_clean":"2024-09-22T04:06:12.951973+0000","last_became_active":"2024-09-22T03:46:55.370620+0000","last_became_peered":"2024-09-22T03:46:55.370620+0000","last_unstale":"2024-09-22T04:06:12.951973+0000","last_undegraded":"2024-09-22T04:06:12.951973+0000","last_fullsized":"2024-09-22T04:06:12.951973+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2163,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.935759+0000","last_change":"2024-09-22T03:46:57.328548+0000","last_active":"2024-09-22T04:06:12.935759+0000","last_peered":"2024-09-22T04:06:12.935759+0000","last_clean":"2024-09-22T04:06:12.935759+0000","last_became_active":"2024-09-22T03:46:55.358980+0000","last_became_peered":"2024-09-22T03:46:55.358980+0000","last_unstale":"2024-09-22T04:06:12.935759+0000","last_undegraded":"2024-09-22T04:06:12.935759+0000","last_fullsized":"2024-09-22T04:06:12.935759+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":2163,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.949682+0000","last_change":"2024-09-22T03:46:57.331270+0000","last_active":"2024-09-22T04:06:12.949682+0000","last_peered":"2024-09-22T04:06:12.949682+0000","last_clean":"2024-09-22T04:06:12.949682+0000","last_became_active":"2024-09-22T03:46:55.383021+0000","last_became_peered":"2024-09-22T03:46:55.383021+0000","last_unstale":"2024-09-22T04:06:12.949682+0000","last_undegraded":"2024-09-22T04:06:12.949682+0000","last_fullsized":"2024-09-22T04:06:12.949682+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2163,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.943350+0000","last_change":"2024-09-22T03:46:57.329346+0000","last_active":"2024-09-22T04:06:12.943350+0000","last_peered":"2024-09-22T04:06:12.943350+0000","last_clean":"2024-09-22T04:06:12.943350+0000","last_became_active":"2024-09-22T03:46:55.358542+0000","last_became_peered":"2024-09-22T03:46:55.358542+0000","last_unstale":"2024-09-22T04:06:12.943350+0000","last_undegraded":"2024-09-22T04:06:12.943350+0000","last_fullsized":"2024-09-22T04:06:12.943350+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2161,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.943416+0000","last_change":"2024-09-22T03:46:57.332056+0000","last_active":"2024-09-22T04:06:12.943416+0000","last_peered":"2024-09-22T04:06:12.943416+0000","last_clean":"2024-09-22T04:06:12.943416+0000","last_became_active":"2024-09-22T03:46:55.412604+0000","last_became_peered":"2024-09-22T03:46:55.412604+0000","last_unstale":"2024-09-22T04:06:12.943416+0000","last_undegraded":"2024-09-22T04:06:12.943416+0000","last_fullsized":"2024-09-22T04:06:12.943416+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":2164,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.958818+0000","last_change":"2024-09-22T03:46:57.335158+0000","last_active":"2024-09-22T04:06:12.958818+0000","last_peered":"2024-09-22T04:06:12.958818+0000","last_clean":"2024-09-22T04:06:12.958818+0000","last_became_active":"2024-09-22T03:46:55.387128+0000","last_became_peered":"2024-09-22T03:46:55.387128+0000","last_unstale":"2024-09-22T04:06:12.958818+0000","last_undegraded":"2024-09-22T04:06:12.958818+0000","last_fullsized":"2024-09-22T04:06:12.958818+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":2235,"reported_epoch":731,"state":"active+clean","last_fresh":"2024-09-22T04:06:12.936476+0000","last_change":"2024-09-22T03:46:53.370744+0000","last_active":"2024-09-22T04:06:12.936476+0000","last_peered":"2024-09-22T04:06:12.936476+0000","last_clean":"2024-09-22T04:06:12.936476+0000","last_became_active":"2024-09-22T03:46:53.349788+0000","last_became_peered":"2024-09-22T03:46:53.349788+0000","last_unstale":"2024-09-22T04:06:12.936476+0000","last_undegraded":"2024-09-22T04:06:12.936476+0000","last_fullsized":"2024-09-22T04:06:12.936476+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:52.275666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:52.275666+0000","last_clean_scrub_stamp":"2024-09-22T03:46:52.275666+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214187458560,"internally_reserved":0,"allocated":24035328,"data_stored":24035328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214187458560,"internally_reserved":0,"allocated":24035328,"data_stored":24035328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53546475520,"internally_reserved":0,"allocated":6397952,"data_stored":6397952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":6248,"kb_used_data":6248,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52291480,"statfs":{"total":53552873472,"available":53546475520,"internally_reserved":0,"allocated":6397952,"data_stored":6397952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":5300,"kb_used_data":5300,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52292428,"statfs":{"total":53552873472,"available":53547446272,"internally_reserved":0,"allocated":5427200,"data_stored":5427200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":5128,"kb_used_data":5128,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52292600,"statfs":{"total":53552873472,"available":53547622400,"internally_reserved":0,"allocated":5251072,"data_stored":5251072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":6796,"kb_used_data":6796,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52290932,"statfs":{"total":53552873472,"available":53545914368,"internally_reserved":0,"allocated":6959104,"data_stored":6959104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53546475520,"internally_reserved":0,"allocated":6397952,"data_stored":6397952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53545914368,"internally_reserved":0,"allocated":6959104,"data_stored":6959104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53547622400,"internally_reserved":0,"allocated":5251072,"data_stored":5251072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53547446272,"internally_reserved":0,"allocated":5427200,"data_stored":5427200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53546475520,"internally_reserved":0,"allocated":6397952,"data_stored":6397952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":0,"total":53552873472,"available":53545914368,"internally_reserved":0,"allocated":6959104,"data_stored":6959104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":1,"total":53552873472,"available":53547622400,"internally_reserved":0,"allocated":5251072,"data_stored":5251072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":2,"total":53552873472,"available":53547446272,"internally_reserved":0,"allocated":5427200,"data_stored":5427200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":3,"total":53552873472,"available":53546475520,"internally_reserved":0,"allocated":6397952,"data_stored":6397952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T04:06:45.849 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-22T04:06:45.850 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T04:06:45.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.957+0000 7f445c0bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:45.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.962+0000 7f445c0bd640 1 Processor -- start 2024-09-22T04:06:45.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.962+0000 7f445c0bd640 1 -- start start 2024-09-22T04:06:45.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.963+0000 7f445c0bd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44540a4930 0x7f445413b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:45.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.963+0000 7f445c0bd640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f445406ec50 con 0x7f44540a3e10 2024-09-22T04:06:45.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.963+0000 7f445c0bd640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f44540a6130 con 0x7f44540a4930 2024-09-22T04:06:45.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.964+0000 7f4459e59640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f44540a3e10 0x7f44540a4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45286/0 (socket says 172.21.15.3:45286) 2024-09-22T04:06:45.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.964+0000 7f4459e59640 1 -- 172.21.15.3:0/4040493487 learned_addr learned my addr 172.21.15.3:0/4040493487 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T04:06:45.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.964+0000 7f4459658640 1 --2- 172.21.15.3:0/4040493487 >> v2:172.21.15.3:3300/0 conn(0x7f44540a4930 0x7f445413b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:45.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.964+0000 7f4458e57640 1 -- 172.21.15.3:0/4040493487 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3751700938 0 0) 0x7f445406ec50 con 0x7f44540a3e10 2024-09-22T04:06:45.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.964+0000 7f4458e57640 1 -- 172.21.15.3:0/4040493487 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f443c003580 con 0x7f44540a3e10 2024-09-22T04:06:45.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.964+0000 7f4459658640 1 -- 172.21.15.3:0/4040493487 >> v1:172.21.15.3:6789/0 conn(0x7f44540a3e10 legacy=0x7f44540a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:45.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.964+0000 7f4459658640 1 -- 172.21.15.3:0/4040493487 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44540a5110 con 0x7f44540a4930 2024-09-22T04:06:45.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.964+0000 7f4459658640 1 --2- 172.21.15.3:0/4040493487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44540a4930 0x7f445413b2d0 crc :-1 s=READY pgs=13009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efb22bf57616178 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:45.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.964+0000 7f4458e57640 1 -- 172.21.15.3:0/4040493487 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f444800e040 con 0x7f44540a4930 2024-09-22T04:06:45.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.964+0000 7f4458e57640 1 -- 172.21.15.3:0/4040493487 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4448007670 con 0x7f44540a4930 2024-09-22T04:06:45.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.964+0000 7f4458e57640 1 -- 172.21.15.3:0/4040493487 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4448007910 con 0x7f44540a4930 2024-09-22T04:06:45.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.965+0000 7f445c0bd640 1 -- 172.21.15.3:0/4040493487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44540a4930 msgr2=0x7f445413b2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:45.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.965+0000 7f445c0bd640 1 --2- 172.21.15.3:0/4040493487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44540a4930 0x7f445413b2d0 crc :-1 s=READY pgs=13009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:45.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.965+0000 7f445c0bd640 1 -- 172.21.15.3:0/4040493487 shutdown_connections 2024-09-22T04:06:45.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.965+0000 7f445c0bd640 1 --2- 172.21.15.3:0/4040493487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44540a4930 0x7f445413b2d0 unknown :-1 s=CLOSED pgs=13009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:45.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.965+0000 7f445c0bd640 1 -- 172.21.15.3:0/4040493487 >> 172.21.15.3:0/4040493487 conn(0x7f44540a74f0 msgr2=0x7f44540a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:45.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.965+0000 7f445c0bd640 1 -- 172.21.15.3:0/4040493487 shutdown_connections 2024-09-22T04:06:45.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.965+0000 7f445c0bd640 1 -- 172.21.15.3:0/4040493487 wait complete. 2024-09-22T04:06:45.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.965+0000 7f445c0bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:45.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.967+0000 7f445c0bd640 1 Processor -- start 2024-09-22T04:06:45.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.967+0000 7f445c0bd640 1 -- start start 2024-09-22T04:06:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.969+0000 7f445c0bd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44540a3e10 0x7f4454123ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.969+0000 7f445c0bd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4454125550 con 0x7f44540a3e10 2024-09-22T04:06:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.969+0000 7f4459e59640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44540a3e10 0x7f4454123ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.969+0000 7f4459e59640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44540a3e10 0x7f4454123ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34464/0 (socket says 172.21.15.3:34464) 2024-09-22T04:06:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.969+0000 7f4459e59640 1 -- 172.21.15.3:0/1107060202 learned_addr learned my addr 172.21.15.3:0/1107060202 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:45.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.969+0000 7f4459e59640 1 -- 172.21.15.3:0/1107060202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4454124d10 con 0x7f44540a3e10 2024-09-22T04:06:45.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.969+0000 7f4459e59640 1 --2- 172.21.15.3:0/1107060202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44540a3e10 0x7f4454123ad0 crc :-1 s=READY pgs=13010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.969+0000 7f44527fc640 1 -- 172.21.15.3:0/1107060202 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f444c005a60 con 0x7f44540a3e10 2024-09-22T04:06:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.969+0000 7f44527fc640 1 -- 172.21.15.3:0/1107060202 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f444c005bc0 con 0x7f44540a3e10 2024-09-22T04:06:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.969+0000 7f445c0bd640 1 -- 172.21.15.3:0/1107060202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4454125e00 con 0x7f44540a3e10 2024-09-22T04:06:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.970+0000 7f44527fc640 1 -- 172.21.15.3:0/1107060202 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f444c005e60 con 0x7f44540a3e10 2024-09-22T04:06:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.970+0000 7f445c0bd640 1 -- 172.21.15.3:0/1107060202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4454126580 con 0x7f44540a3e10 2024-09-22T04:06:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.970+0000 7f44527fc640 1 -- 172.21.15.3:0/1107060202 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f444c0070b0 con 0x7f44540a3e10 2024-09-22T04:06:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.970+0000 7f445c0bd640 1 -- 172.21.15.3:0/1107060202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44541279a0 con 0x7f44540a3e10 2024-09-22T04:06:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.971+0000 7f44527fc640 1 --2- 172.21.15.3:0/1107060202 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f444003ead0 0x7f4440040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:45.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.971+0000 7f44527fc640 1 -- 172.21.15.3:0/1107060202 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(733..733 src has 590..733) ==== 3431+0+0 (crc 0 0 0) 0x7f444c04fbe0 con 0x7f44540a3e10 2024-09-22T04:06:45.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.971+0000 7f4459658640 1 --2- 172.21.15.3:0/1107060202 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f444003ead0 0x7f4440040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:45.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.972+0000 7f4459658640 1 --2- 172.21.15.3:0/1107060202 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f444003ead0 0x7f4440040fe0 crc :-1 s=READY pgs=3871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:45.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:45.976+0000 7f44527fc640 1 -- 172.21.15.3:0/1107060202 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f444c00dc60 con 0x7f44540a3e10 2024-09-22T04:06:46.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.160+0000 7f445c0bd640 1 -- 172.21.15.3:0/1107060202 --> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4454073410 con 0x7f444003ead0 2024-09-22T04:06:46.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.163+0000 7f44527fc640 1 -- 172.21.15.3:0/1107060202 <== mgr.4099 v2:172.21.15.123:6800/551784226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+109430 (crc 0 0 0) 0x7f4454073410 con 0x7f444003ead0 2024-09-22T04:06:46.165 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T04:06:46.165 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-22T04:06:46.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.165+0000 7f445c0bd640 1 -- 172.21.15.3:0/1107060202 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f444003ead0 msgr2=0x7f4440040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:46.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.165+0000 7f445c0bd640 1 --2- 172.21.15.3:0/1107060202 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f444003ead0 0x7f4440040fe0 crc :-1 s=READY pgs=3871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:46.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.165+0000 7f445c0bd640 1 -- 172.21.15.3:0/1107060202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44540a3e10 msgr2=0x7f4454123ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:46.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.165+0000 7f445c0bd640 1 --2- 172.21.15.3:0/1107060202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44540a3e10 0x7f4454123ad0 crc :-1 s=READY pgs=13010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:46.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.165+0000 7f445c0bd640 1 -- 172.21.15.3:0/1107060202 shutdown_connections 2024-09-22T04:06:46.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.165+0000 7f445c0bd640 1 --2- 172.21.15.3:0/1107060202 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f444003ead0 0x7f4440040fe0 unknown :-1 s=CLOSED pgs=3871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:46.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.165+0000 7f445c0bd640 1 --2- 172.21.15.3:0/1107060202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44540a3e10 0x7f4454123ad0 unknown :-1 s=CLOSED pgs=13010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:46.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.165+0000 7f445c0bd640 1 -- 172.21.15.3:0/1107060202 >> 172.21.15.3:0/1107060202 conn(0x7f44540a74f0 msgr2=0x7f4454071360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:46.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.165+0000 7f445c0bd640 1 -- 172.21.15.3:0/1107060202 shutdown_connections 2024-09-22T04:06:46.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.165+0000 7f445c0bd640 1 -- 172.21.15.3:0/1107060202 wait complete. 2024-09-22T04:06:46.178 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":1325,"stamp":"2024-09-22T04:06:45.841677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590392,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":52,"num_write":88,"num_write_kb":1115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62,"ondisk_log_size":62,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":17344,"kb_used_data":17344,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209173568,"statfs":{"total":214211493888,"available":214193733632,"internally_reserved":0,"allocated":17760256,"data_stored":17760256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-321875,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-25,"num_write_kb":-325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001972"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.468761+0000","last_change":"2024-09-22T03:52:13.461984+0000","last_active":"2024-09-22T04:06:44.468761+0000","last_peered":"2024-09-22T04:06:44.468761+0000","last_clean":"2024-09-22T04:06:44.468761+0000","last_became_active":"2024-09-22T03:52:13.436104+0000","last_became_peered":"2024-09-22T03:52:13.436104+0000","last_unstale":"2024-09-22T04:06:44.468761+0000","last_undegraded":"2024-09-22T04:06:44.468761+0000","last_fullsized":"2024-09-22T04:06:44.468761+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.531561+0000","last_change":"2024-09-22T03:52:13.517533+0000","last_active":"2024-09-22T04:06:44.531561+0000","last_peered":"2024-09-22T04:06:44.531561+0000","last_clean":"2024-09-22T04:06:44.531561+0000","last_became_active":"2024-09-22T03:52:13.485941+0000","last_became_peered":"2024-09-22T03:52:13.485941+0000","last_unstale":"2024-09-22T04:06:44.531561+0000","last_undegraded":"2024-09-22T04:06:44.531561+0000","last_fullsized":"2024-09-22T04:06:44.531561+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.476678+0000","last_change":"2024-09-22T03:52:13.441983+0000","last_active":"2024-09-22T04:06:44.476678+0000","last_peered":"2024-09-22T04:06:44.476678+0000","last_clean":"2024-09-22T04:06:44.476678+0000","last_became_active":"2024-09-22T03:52:13.435054+0000","last_became_peered":"2024-09-22T03:52:13.435054+0000","last_unstale":"2024-09-22T04:06:44.476678+0000","last_undegraded":"2024-09-22T04:06:44.476678+0000","last_fullsized":"2024-09-22T04:06:44.476678+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.484441+0000","last_change":"2024-09-22T03:52:13.475862+0000","last_active":"2024-09-22T04:06:44.484441+0000","last_peered":"2024-09-22T04:06:44.484441+0000","last_clean":"2024-09-22T04:06:44.484441+0000","last_became_active":"2024-09-22T03:52:13.424125+0000","last_became_peered":"2024-09-22T03:52:13.424125+0000","last_unstale":"2024-09-22T04:06:44.484441+0000","last_undegraded":"2024-09-22T04:06:44.484441+0000","last_fullsized":"2024-09-22T04:06:44.484441+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.489151+0000","last_change":"2024-09-22T03:52:13.638357+0000","last_active":"2024-09-22T04:06:44.489151+0000","last_peered":"2024-09-22T04:06:44.489151+0000","last_clean":"2024-09-22T04:06:44.489151+0000","last_became_active":"2024-09-22T03:52:13.612276+0000","last_became_peered":"2024-09-22T03:52:13.612276+0000","last_unstale":"2024-09-22T04:06:44.489151+0000","last_undegraded":"2024-09-22T04:06:44.489151+0000","last_fullsized":"2024-09-22T04:06:44.489151+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.482802+0000","last_change":"2024-09-22T03:52:13.631942+0000","last_active":"2024-09-22T04:06:44.482802+0000","last_peered":"2024-09-22T04:06:44.482802+0000","last_clean":"2024-09-22T04:06:44.482802+0000","last_became_active":"2024-09-22T03:52:13.584449+0000","last_became_peered":"2024-09-22T03:52:13.584449+0000","last_unstale":"2024-09-22T04:06:44.482802+0000","last_undegraded":"2024-09-22T04:06:44.482802+0000","last_fullsized":"2024-09-22T04:06:44.482802+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.492811+0000","last_change":"2024-09-22T03:52:13.462393+0000","last_active":"2024-09-22T04:06:44.492811+0000","last_peered":"2024-09-22T04:06:44.492811+0000","last_clean":"2024-09-22T04:06:44.492811+0000","last_became_active":"2024-09-22T03:52:13.438920+0000","last_became_peered":"2024-09-22T03:52:13.438920+0000","last_unstale":"2024-09-22T04:06:44.492811+0000","last_undegraded":"2024-09-22T04:06:44.492811+0000","last_fullsized":"2024-09-22T04:06:44.492811+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.481246+0000","last_change":"2024-09-22T03:52:13.435067+0000","last_active":"2024-09-22T04:06:44.481246+0000","last_peered":"2024-09-22T04:06:44.481246+0000","last_clean":"2024-09-22T04:06:44.481246+0000","last_became_active":"2024-09-22T03:52:13.424399+0000","last_became_peered":"2024-09-22T03:52:13.424399+0000","last_unstale":"2024-09-22T04:06:44.481246+0000","last_undegraded":"2024-09-22T04:06:44.481246+0000","last_fullsized":"2024-09-22T04:06:44.481246+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.456201+0000","last_change":"2024-09-22T03:52:13.414909+0000","last_active":"2024-09-22T04:06:44.456201+0000","last_peered":"2024-09-22T04:06:44.456201+0000","last_clean":"2024-09-22T04:06:44.456201+0000","last_became_active":"2024-09-22T03:52:13.367950+0000","last_became_peered":"2024-09-22T03:52:13.367950+0000","last_unstale":"2024-09-22T04:06:44.456201+0000","last_undegraded":"2024-09-22T04:06:44.456201+0000","last_fullsized":"2024-09-22T04:06:44.456201+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.463555+0000","last_change":"2024-09-22T03:52:13.421431+0000","last_active":"2024-09-22T04:06:44.463555+0000","last_peered":"2024-09-22T04:06:44.463555+0000","last_clean":"2024-09-22T04:06:44.463555+0000","last_became_active":"2024-09-22T03:52:13.370886+0000","last_became_peered":"2024-09-22T03:52:13.370886+0000","last_unstale":"2024-09-22T04:06:44.463555+0000","last_undegraded":"2024-09-22T04:06:44.463555+0000","last_fullsized":"2024-09-22T04:06:44.463555+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.469081+0000","last_change":"2024-09-22T03:52:13.488521+0000","last_active":"2024-09-22T04:06:44.469081+0000","last_peered":"2024-09-22T04:06:44.469081+0000","last_clean":"2024-09-22T04:06:44.469081+0000","last_became_active":"2024-09-22T03:52:13.477618+0000","last_became_peered":"2024-09-22T03:52:13.477618+0000","last_unstale":"2024-09-22T04:06:44.469081+0000","last_undegraded":"2024-09-22T04:06:44.469081+0000","last_fullsized":"2024-09-22T04:06:44.469081+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.477943+0000","last_change":"2024-09-22T03:52:13.423108+0000","last_active":"2024-09-22T04:06:44.477943+0000","last_peered":"2024-09-22T04:06:44.477943+0000","last_clean":"2024-09-22T04:06:44.477943+0000","last_became_active":"2024-09-22T03:52:13.354826+0000","last_became_peered":"2024-09-22T03:52:13.354826+0000","last_unstale":"2024-09-22T04:06:44.477943+0000","last_undegraded":"2024-09-22T04:06:44.477943+0000","last_fullsized":"2024-09-22T04:06:44.477943+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.461834+0000","last_change":"2024-09-22T03:52:13.456498+0000","last_active":"2024-09-22T04:06:44.461834+0000","last_peered":"2024-09-22T04:06:44.461834+0000","last_clean":"2024-09-22T04:06:44.461834+0000","last_became_active":"2024-09-22T03:52:13.405184+0000","last_became_peered":"2024-09-22T03:52:13.405184+0000","last_unstale":"2024-09-22T04:06:44.461834+0000","last_undegraded":"2024-09-22T04:06:44.461834+0000","last_fullsized":"2024-09-22T04:06:44.461834+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.477816+0000","last_change":"2024-09-22T03:52:13.548435+0000","last_active":"2024-09-22T04:06:44.477816+0000","last_peered":"2024-09-22T04:06:44.477816+0000","last_clean":"2024-09-22T04:06:44.477816+0000","last_became_active":"2024-09-22T03:52:13.533286+0000","last_became_peered":"2024-09-22T03:52:13.533286+0000","last_unstale":"2024-09-22T04:06:44.477816+0000","last_undegraded":"2024-09-22T04:06:44.477816+0000","last_fullsized":"2024-09-22T04:06:44.477816+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.476662+0000","last_change":"2024-09-22T03:52:13.555314+0000","last_active":"2024-09-22T04:06:44.476662+0000","last_peered":"2024-09-22T04:06:44.476662+0000","last_clean":"2024-09-22T04:06:44.476662+0000","last_became_active":"2024-09-22T03:52:13.531887+0000","last_became_peered":"2024-09-22T03:52:13.531887+0000","last_unstale":"2024-09-22T04:06:44.476662+0000","last_undegraded":"2024-09-22T04:06:44.476662+0000","last_fullsized":"2024-09-22T04:06:44.476662+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.538057+0000","last_change":"2024-09-22T03:52:13.576998+0000","last_active":"2024-09-22T04:06:44.538057+0000","last_peered":"2024-09-22T04:06:44.538057+0000","last_clean":"2024-09-22T04:06:44.538057+0000","last_became_active":"2024-09-22T03:52:13.562231+0000","last_became_peered":"2024-09-22T03:52:13.562231+0000","last_unstale":"2024-09-22T04:06:44.538057+0000","last_undegraded":"2024-09-22T04:06:44.538057+0000","last_fullsized":"2024-09-22T04:06:44.538057+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.480181+0000","last_change":"2024-09-22T03:52:13.527047+0000","last_active":"2024-09-22T04:06:44.480181+0000","last_peered":"2024-09-22T04:06:44.480181+0000","last_clean":"2024-09-22T04:06:44.480181+0000","last_became_active":"2024-09-22T03:52:13.496766+0000","last_became_peered":"2024-09-22T03:52:13.496766+0000","last_unstale":"2024-09-22T04:06:44.480181+0000","last_undegraded":"2024-09-22T04:06:44.480181+0000","last_fullsized":"2024-09-22T04:06:44.480181+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.482475+0000","last_change":"2024-09-22T03:52:13.517430+0000","last_active":"2024-09-22T04:06:44.482475+0000","last_peered":"2024-09-22T04:06:44.482475+0000","last_clean":"2024-09-22T04:06:44.482475+0000","last_became_active":"2024-09-22T03:52:13.496886+0000","last_became_peered":"2024-09-22T03:52:13.496886+0000","last_unstale":"2024-09-22T04:06:44.482475+0000","last_undegraded":"2024-09-22T04:06:44.482475+0000","last_fullsized":"2024-09-22T04:06:44.482475+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.490055+0000","last_change":"2024-09-22T03:52:13.464870+0000","last_active":"2024-09-22T04:06:44.490055+0000","last_peered":"2024-09-22T04:06:44.490055+0000","last_clean":"2024-09-22T04:06:44.490055+0000","last_became_active":"2024-09-22T03:52:13.412673+0000","last_became_peered":"2024-09-22T03:52:13.412673+0000","last_unstale":"2024-09-22T04:06:44.490055+0000","last_undegraded":"2024-09-22T04:06:44.490055+0000","last_fullsized":"2024-09-22T04:06:44.490055+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.478723+0000","last_change":"2024-09-22T03:52:13.477357+0000","last_active":"2024-09-22T04:06:44.478723+0000","last_peered":"2024-09-22T04:06:44.478723+0000","last_clean":"2024-09-22T04:06:44.478723+0000","last_became_active":"2024-09-22T03:52:13.441995+0000","last_became_peered":"2024-09-22T03:52:13.441995+0000","last_unstale":"2024-09-22T04:06:44.478723+0000","last_undegraded":"2024-09-22T04:06:44.478723+0000","last_fullsized":"2024-09-22T04:06:44.478723+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.463150+0000","last_change":"2024-09-22T03:52:13.667385+0000","last_active":"2024-09-22T04:06:44.463150+0000","last_peered":"2024-09-22T04:06:44.463150+0000","last_clean":"2024-09-22T04:06:44.463150+0000","last_became_active":"2024-09-22T03:52:13.657594+0000","last_became_peered":"2024-09-22T03:52:13.657594+0000","last_unstale":"2024-09-22T04:06:44.463150+0000","last_undegraded":"2024-09-22T04:06:44.463150+0000","last_fullsized":"2024-09-22T04:06:44.463150+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.474456+0000","last_change":"2024-09-22T03:52:13.488158+0000","last_active":"2024-09-22T04:06:44.474456+0000","last_peered":"2024-09-22T04:06:44.474456+0000","last_clean":"2024-09-22T04:06:44.474456+0000","last_became_active":"2024-09-22T03:52:13.435529+0000","last_became_peered":"2024-09-22T03:52:13.435529+0000","last_unstale":"2024-09-22T04:06:44.474456+0000","last_undegraded":"2024-09-22T04:06:44.474456+0000","last_fullsized":"2024-09-22T04:06:44.474456+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.477411+0000","last_change":"2024-09-22T03:52:13.418644+0000","last_active":"2024-09-22T04:06:44.477411+0000","last_peered":"2024-09-22T04:06:44.477411+0000","last_clean":"2024-09-22T04:06:44.477411+0000","last_became_active":"2024-09-22T03:52:13.411413+0000","last_became_peered":"2024-09-22T03:52:13.411413+0000","last_unstale":"2024-09-22T04:06:44.477411+0000","last_undegraded":"2024-09-22T04:06:44.477411+0000","last_fullsized":"2024-09-22T04:06:44.477411+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.465248+0000","last_change":"2024-09-22T03:52:13.441353+0000","last_active":"2024-09-22T04:06:44.465248+0000","last_peered":"2024-09-22T04:06:44.465248+0000","last_clean":"2024-09-22T04:06:44.465248+0000","last_became_active":"2024-09-22T03:52:13.399486+0000","last_became_peered":"2024-09-22T03:52:13.399486+0000","last_unstale":"2024-09-22T04:06:44.465248+0000","last_undegraded":"2024-09-22T04:06:44.465248+0000","last_fullsized":"2024-09-22T04:06:44.465248+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.463692+0000","last_change":"2024-09-22T03:52:13.359382+0000","last_active":"2024-09-22T04:06:44.463692+0000","last_peered":"2024-09-22T04:06:44.463692+0000","last_clean":"2024-09-22T04:06:44.463692+0000","last_became_active":"2024-09-22T03:52:13.335952+0000","last_became_peered":"2024-09-22T03:52:13.335952+0000","last_unstale":"2024-09-22T04:06:44.463692+0000","last_undegraded":"2024-09-22T04:06:44.463692+0000","last_fullsized":"2024-09-22T04:06:44.463692+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.498735+0000","last_change":"2024-09-22T03:52:13.445595+0000","last_active":"2024-09-22T04:06:44.498735+0000","last_peered":"2024-09-22T04:06:44.498735+0000","last_clean":"2024-09-22T04:06:44.498735+0000","last_became_active":"2024-09-22T03:52:13.414407+0000","last_became_peered":"2024-09-22T03:52:13.414407+0000","last_unstale":"2024-09-22T04:06:44.498735+0000","last_undegraded":"2024-09-22T04:06:44.498735+0000","last_fullsized":"2024-09-22T04:06:44.498735+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.482567+0000","last_change":"2024-09-22T03:52:13.491741+0000","last_active":"2024-09-22T04:06:44.482567+0000","last_peered":"2024-09-22T04:06:44.482567+0000","last_clean":"2024-09-22T04:06:44.482567+0000","last_became_active":"2024-09-22T03:52:13.431192+0000","last_became_peered":"2024-09-22T03:52:13.431192+0000","last_unstale":"2024-09-22T04:06:44.482567+0000","last_undegraded":"2024-09-22T04:06:44.482567+0000","last_fullsized":"2024-09-22T04:06:44.482567+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.464658+0000","last_change":"2024-09-22T03:52:13.501197+0000","last_active":"2024-09-22T04:06:44.464658+0000","last_peered":"2024-09-22T04:06:44.464658+0000","last_clean":"2024-09-22T04:06:44.464658+0000","last_became_active":"2024-09-22T03:52:13.393363+0000","last_became_peered":"2024-09-22T03:52:13.393363+0000","last_unstale":"2024-09-22T04:06:44.464658+0000","last_undegraded":"2024-09-22T04:06:44.464658+0000","last_fullsized":"2024-09-22T04:06:44.464658+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.490609+0000","last_change":"2024-09-22T03:52:13.454940+0000","last_active":"2024-09-22T04:06:44.490609+0000","last_peered":"2024-09-22T04:06:44.490609+0000","last_clean":"2024-09-22T04:06:44.490609+0000","last_became_active":"2024-09-22T03:52:13.423424+0000","last_became_peered":"2024-09-22T03:52:13.423424+0000","last_unstale":"2024-09-22T04:06:44.490609+0000","last_undegraded":"2024-09-22T04:06:44.490609+0000","last_fullsized":"2024-09-22T04:06:44.490609+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.474681+0000","last_change":"2024-09-22T03:52:13.491666+0000","last_active":"2024-09-22T04:06:44.474681+0000","last_peered":"2024-09-22T04:06:44.474681+0000","last_clean":"2024-09-22T04:06:44.474681+0000","last_became_active":"2024-09-22T03:52:13.404954+0000","last_became_peered":"2024-09-22T03:52:13.404954+0000","last_unstale":"2024-09-22T04:06:44.474681+0000","last_undegraded":"2024-09-22T04:06:44.474681+0000","last_fullsized":"2024-09-22T04:06:44.474681+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"241'7","reported_seq":1497,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.472070+0000","last_change":"2024-09-22T03:52:13.469238+0000","last_active":"2024-09-22T04:06:44.472070+0000","last_peered":"2024-09-22T04:06:44.472070+0000","last_clean":"2024-09-22T04:06:44.472070+0000","last_became_active":"2024-09-22T03:52:13.422437+0000","last_became_peered":"2024-09-22T03:52:13.422437+0000","last_unstale":"2024-09-22T04:06:44.472070+0000","last_undegraded":"2024-09-22T04:06:44.472070+0000","last_fullsized":"2024-09-22T04:06:44.472070+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.491142+0000","last_change":"2024-09-22T03:52:13.460700+0000","last_active":"2024-09-22T04:06:44.491142+0000","last_peered":"2024-09-22T04:06:44.491142+0000","last_clean":"2024-09-22T04:06:44.491142+0000","last_became_active":"2024-09-22T03:52:13.420437+0000","last_became_peered":"2024-09-22T03:52:13.420437+0000","last_unstale":"2024-09-22T04:06:44.491142+0000","last_undegraded":"2024-09-22T04:06:44.491142+0000","last_fullsized":"2024-09-22T04:06:44.491142+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":2169,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.488571+0000","last_change":"2024-09-22T03:46:57.335250+0000","last_active":"2024-09-22T04:06:44.488571+0000","last_peered":"2024-09-22T04:06:44.488571+0000","last_clean":"2024-09-22T04:06:44.488571+0000","last_became_active":"2024-09-22T03:46:55.378262+0000","last_became_peered":"2024-09-22T03:46:55.378262+0000","last_unstale":"2024-09-22T04:06:44.488571+0000","last_undegraded":"2024-09-22T04:06:44.488571+0000","last_fullsized":"2024-09-22T04:06:44.488571+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2169,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.481104+0000","last_change":"2024-09-22T03:46:57.333626+0000","last_active":"2024-09-22T04:06:44.481104+0000","last_peered":"2024-09-22T04:06:44.481104+0000","last_clean":"2024-09-22T04:06:44.481104+0000","last_became_active":"2024-09-22T03:46:55.378604+0000","last_became_peered":"2024-09-22T03:46:55.378604+0000","last_unstale":"2024-09-22T04:06:44.481104+0000","last_undegraded":"2024-09-22T04:06:44.481104+0000","last_fullsized":"2024-09-22T04:06:44.481104+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2169,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.483589+0000","last_change":"2024-09-22T03:46:57.329704+0000","last_active":"2024-09-22T04:06:44.483589+0000","last_peered":"2024-09-22T04:06:44.483589+0000","last_clean":"2024-09-22T04:06:44.483589+0000","last_became_active":"2024-09-22T03:46:55.370620+0000","last_became_peered":"2024-09-22T03:46:55.370620+0000","last_unstale":"2024-09-22T04:06:44.483589+0000","last_undegraded":"2024-09-22T04:06:44.483589+0000","last_fullsized":"2024-09-22T04:06:44.483589+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2169,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.470822+0000","last_change":"2024-09-22T03:46:57.328548+0000","last_active":"2024-09-22T04:06:44.470822+0000","last_peered":"2024-09-22T04:06:44.470822+0000","last_clean":"2024-09-22T04:06:44.470822+0000","last_became_active":"2024-09-22T03:46:55.358980+0000","last_became_peered":"2024-09-22T03:46:55.358980+0000","last_unstale":"2024-09-22T04:06:44.470822+0000","last_undegraded":"2024-09-22T04:06:44.470822+0000","last_fullsized":"2024-09-22T04:06:44.470822+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":2169,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.492255+0000","last_change":"2024-09-22T03:46:57.331270+0000","last_active":"2024-09-22T04:06:44.492255+0000","last_peered":"2024-09-22T04:06:44.492255+0000","last_clean":"2024-09-22T04:06:44.492255+0000","last_became_active":"2024-09-22T03:46:55.383021+0000","last_became_peered":"2024-09-22T03:46:55.383021+0000","last_unstale":"2024-09-22T04:06:44.492255+0000","last_undegraded":"2024-09-22T04:06:44.492255+0000","last_fullsized":"2024-09-22T04:06:44.492255+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2169,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.466290+0000","last_change":"2024-09-22T03:46:57.329346+0000","last_active":"2024-09-22T04:06:44.466290+0000","last_peered":"2024-09-22T04:06:44.466290+0000","last_clean":"2024-09-22T04:06:44.466290+0000","last_became_active":"2024-09-22T03:46:55.358542+0000","last_became_peered":"2024-09-22T03:46:55.358542+0000","last_unstale":"2024-09-22T04:06:44.466290+0000","last_undegraded":"2024-09-22T04:06:44.466290+0000","last_fullsized":"2024-09-22T04:06:44.466290+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2167,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.478791+0000","last_change":"2024-09-22T03:46:57.332056+0000","last_active":"2024-09-22T04:06:44.478791+0000","last_peered":"2024-09-22T04:06:44.478791+0000","last_clean":"2024-09-22T04:06:44.478791+0000","last_became_active":"2024-09-22T03:46:55.412604+0000","last_became_peered":"2024-09-22T03:46:55.412604+0000","last_unstale":"2024-09-22T04:06:44.478791+0000","last_undegraded":"2024-09-22T04:06:44.478791+0000","last_fullsized":"2024-09-22T04:06:44.478791+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":2170,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.487146+0000","last_change":"2024-09-22T03:46:57.335158+0000","last_active":"2024-09-22T04:06:44.487146+0000","last_peered":"2024-09-22T04:06:44.487146+0000","last_clean":"2024-09-22T04:06:44.487146+0000","last_became_active":"2024-09-22T03:46:55.387128+0000","last_became_peered":"2024-09-22T03:46:55.387128+0000","last_unstale":"2024-09-22T04:06:44.487146+0000","last_undegraded":"2024-09-22T04:06:44.487146+0000","last_fullsized":"2024-09-22T04:06:44.487146+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":2241,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.458607+0000","last_change":"2024-09-22T03:46:53.370744+0000","last_active":"2024-09-22T04:06:44.458607+0000","last_peered":"2024-09-22T04:06:44.458607+0000","last_clean":"2024-09-22T04:06:44.458607+0000","last_became_active":"2024-09-22T03:46:53.349788+0000","last_became_peered":"2024-09-22T03:46:53.349788+0000","last_unstale":"2024-09-22T04:06:44.458607+0000","last_undegraded":"2024-09-22T04:06:44.458607+0000","last_fullsized":"2024-09-22T04:06:44.458607+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:52.275666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:52.275666+0000","last_clean_scrub_stamp":"2024-09-22T03:46:52.275666+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214194618368,"internally_reserved":0,"allocated":16875520,"data_stored":16875520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214194618368,"internally_reserved":0,"allocated":16875520,"data_stored":16875520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53548302336,"internally_reserved":0,"allocated":4571136,"data_stored":4571136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706858,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":4464,"kb_used_data":4464,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52293264,"statfs":{"total":53552873472,"available":53548302336,"internally_reserved":0,"allocated":4571136,"data_stored":4571136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706858,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":3596,"kb_used_data":3596,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52294132,"statfs":{"total":53552873472,"available":53549191168,"internally_reserved":0,"allocated":3682304,"data_stored":3682304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706858,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":3568,"kb_used_data":3568,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52294160,"statfs":{"total":53552873472,"available":53549219840,"internally_reserved":0,"allocated":3653632,"data_stored":3653632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706857,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":5716,"kb_used_data":5716,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52292012,"statfs":{"total":53552873472,"available":53547020288,"internally_reserved":0,"allocated":5853184,"data_stored":5853184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53548302336,"internally_reserved":0,"allocated":4571136,"data_stored":4571136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53547905024,"internally_reserved":0,"allocated":4968448,"data_stored":4968448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53549219840,"internally_reserved":0,"allocated":3653632,"data_stored":3653632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53549191168,"internally_reserved":0,"allocated":3682304,"data_stored":3682304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53548302336,"internally_reserved":0,"allocated":4571136,"data_stored":4571136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":0,"total":53552873472,"available":53547905024,"internally_reserved":0,"allocated":4968448,"data_stored":4968448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":1,"total":53552873472,"available":53549219840,"internally_reserved":0,"allocated":3653632,"data_stored":3653632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":2,"total":53552873472,"available":53549191168,"internally_reserved":0,"allocated":3682304,"data_stored":3682304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":3,"total":53552873472,"available":53548302336,"internally_reserved":0,"allocated":4571136,"data_stored":4571136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T04:06:46.180 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T04:06:46.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.288+0000 7fbad547d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.293+0000 7fbad547d640 1 Processor -- start 2024-09-22T04:06:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.293+0000 7fbad547d640 1 -- start start 2024-09-22T04:06:46.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.295+0000 7fbad547d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbad006f2d0 0x7fbad013ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:46.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.295+0000 7fbad547d640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbad00a11d0 con 0x7fbad006e7b0 2024-09-22T04:06:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.295+0000 7fbad547d640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbad00a2a20 con 0x7fbad006f2d0 2024-09-22T04:06:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.295+0000 7fbace7fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbad006f2d0 0x7fbad013ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.295+0000 7fbaceffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbad006e7b0 0x7fbad006ebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45302/0 (socket says 172.21.15.3:45302) 2024-09-22T04:06:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.295+0000 7fbace7fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbad006f2d0 0x7fbad013ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34480/0 (socket says 172.21.15.3:34480) 2024-09-22T04:06:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.295+0000 7fbaceffd640 1 -- 172.21.15.3:0/1757046648 learned_addr learned my addr 172.21.15.3:0/1757046648 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T04:06:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.295+0000 7fbace7fc640 1 -- 172.21.15.3:0/1757046648 >> v1:172.21.15.3:6789/0 conn(0x7fbad006e7b0 legacy=0x7fbad006ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.295+0000 7fbacdffb640 1 -- 172.21.15.3:0/1757046648 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1326880281 0 0) 0x7fbad00a11d0 con 0x7fbad006e7b0 2024-09-22T04:06:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.295+0000 7fbace7fc640 1 -- 172.21.15.3:0/1757046648 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad00a0df0 con 0x7fbad006f2d0 2024-09-22T04:06:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.295+0000 7fbace7fc640 1 --2- 172.21.15.3:0/1757046648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad006f2d0 0x7fbad013ae40 crc :-1 s=READY pgs=13012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7b2c4f034b10f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.295+0000 7fbacdffb640 1 -- 172.21.15.3:0/1757046648 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbac4009d90 con 0x7fbad006f2d0 2024-09-22T04:06:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.295+0000 7fbacdffb640 1 -- 172.21.15.3:0/1757046648 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbac400e230 con 0x7fbad006f2d0 2024-09-22T04:06:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.295+0000 7fbacdffb640 1 -- 172.21.15.3:0/1757046648 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbac400e4d0 con 0x7fbad006f2d0 2024-09-22T04:06:46.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.296+0000 7fbad547d640 1 -- 172.21.15.3:0/1757046648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad006f2d0 msgr2=0x7fbad013ae40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:46.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.296+0000 7fbad547d640 1 --2- 172.21.15.3:0/1757046648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad006f2d0 0x7fbad013ae40 crc :-1 s=READY pgs=13012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:46.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.296+0000 7fbad547d640 1 -- 172.21.15.3:0/1757046648 shutdown_connections 2024-09-22T04:06:46.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.296+0000 7fbad547d640 1 --2- 172.21.15.3:0/1757046648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad006f2d0 0x7fbad013ae40 unknown :-1 s=CLOSED pgs=13012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:46.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.296+0000 7fbad547d640 1 -- 172.21.15.3:0/1757046648 >> 172.21.15.3:0/1757046648 conn(0x7fbad00a40f0 msgr2=0x7fbad00a4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:46.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.296+0000 7fbad547d640 1 -- 172.21.15.3:0/1757046648 shutdown_connections 2024-09-22T04:06:46.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.296+0000 7fbad547d640 1 -- 172.21.15.3:0/1757046648 wait complete. 2024-09-22T04:06:46.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.296+0000 7fbad547d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:46.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.298+0000 7fbad547d640 1 Processor -- start 2024-09-22T04:06:46.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.298+0000 7fbad547d640 1 -- start start 2024-09-22T04:06:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.300+0000 7fbad547d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad006e7b0 0x7fbad01336a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.300+0000 7fbad547d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbad011f980 con 0x7fbad006e7b0 2024-09-22T04:06:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.300+0000 7fbaceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad006e7b0 0x7fbad01336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.300+0000 7fbaceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad006e7b0 0x7fbad01336a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34484/0 (socket says 172.21.15.3:34484) 2024-09-22T04:06:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.300+0000 7fbaceffd640 1 -- 172.21.15.3:0/2758455375 learned_addr learned my addr 172.21.15.3:0/2758455375 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.300+0000 7fbaceffd640 1 -- 172.21.15.3:0/2758455375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad011ff60 con 0x7fbad006e7b0 2024-09-22T04:06:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.301+0000 7fbaceffd640 1 --2- 172.21.15.3:0/2758455375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad006e7b0 0x7fbad01336a0 crc :-1 s=READY pgs=13013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.301+0000 7fbab77fe640 1 -- 172.21.15.3:0/2758455375 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbac0004b80 con 0x7fbad006e7b0 2024-09-22T04:06:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.301+0000 7fbab77fe640 1 -- 172.21.15.3:0/2758455375 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fbac0004ce0 con 0x7fbad006e7b0 2024-09-22T04:06:46.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.301+0000 7fbad547d640 1 -- 172.21.15.3:0/2758455375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad011f670 con 0x7fbad006e7b0 2024-09-22T04:06:46.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.301+0000 7fbab77fe640 1 -- 172.21.15.3:0/2758455375 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fbac0007120 con 0x7fbad006e7b0 2024-09-22T04:06:46.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.301+0000 7fbad547d640 1 -- 172.21.15.3:0/2758455375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad01207b0 con 0x7fbad006e7b0 2024-09-22T04:06:46.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.302+0000 7fbad547d640 1 -- 172.21.15.3:0/2758455375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad006ebc0 con 0x7fbad006e7b0 2024-09-22T04:06:46.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.302+0000 7fbab77fe640 1 -- 172.21.15.3:0/2758455375 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fbac00072d0 con 0x7fbad006e7b0 2024-09-22T04:06:46.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.302+0000 7fbab77fe640 1 --2- 172.21.15.3:0/2758455375 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbabc03ead0 0x7fbabc040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:46.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.302+0000 7fbab77fe640 1 -- 172.21.15.3:0/2758455375 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(733..733 src has 590..733) ==== 3431+0+0 (crc 0 0 0) 0x7fbac004fcc0 con 0x7fbad006e7b0 2024-09-22T04:06:46.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.303+0000 7fbace7fc640 1 --2- 172.21.15.3:0/2758455375 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbabc03ead0 0x7fbabc040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:46.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.303+0000 7fbace7fc640 1 --2- 172.21.15.3:0/2758455375 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbabc03ead0 0x7fbabc040fe0 crc :-1 s=READY pgs=3872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:46.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.307+0000 7fbab77fe640 1 -- 172.21.15.3:0/2758455375 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7fbac000f230 con 0x7fbad006e7b0 2024-09-22T04:06:46.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.496+0000 7fbad547d640 1 -- 172.21.15.3:0/2758455375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbad00a4f60 con 0x7fbad006e7b0 2024-09-22T04:06:46.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.497+0000 7fbab77fe640 1 -- 172.21.15.3:0/2758455375 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v733) ==== 74+0+11255 (crc 0 0 0) 0x7fbac0006740 con 0x7fbad006e7b0 2024-09-22T04:06:46.498 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T04:06:46.498 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":733,"fsid":"012c124d-1263-4b17-8b29-e14858734658","created":"2024-09-22T03:46:43.165321+0000","modified":"2024-09-22T04:06:44.421131+0000","last_up_change":"2024-09-22T03:46:50.268789+0000","last_in_change":"2024-09-22T03:46:45.294435+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":200,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T03:46:51.582570+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T03:46:53.510167+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":75,"pool_name":"ceph_test_rados_api_asio","create_time":"2024-09-22T03:52:11.288492+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"242","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae27f46d-f8f7-4336-ad5e-ef956da82898","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":720,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6801","nonce":2889521405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2889521405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6805","nonce":2889521405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":2889521405}]},"public_addr":"172.21.15.3:6801/2889521405","cluster_addr":"172.21.15.3:6800/2889521405","heartbeat_back_addr":"172.21.15.3:6805/2889521405","heartbeat_front_addr":"172.21.15.3:6804/2889521405","state":["exists","up"]},{"osd":1,"uuid":"d04614be-cd17-47ac-a9ec-7708527db672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":720,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":2784566484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6803","nonce":2784566484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6807","nonce":2784566484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":2784566484}]},"public_addr":"172.21.15.3:6802/2784566484","cluster_addr":"172.21.15.3:6803/2784566484","heartbeat_back_addr":"172.21.15.3:6807/2784566484","heartbeat_front_addr":"172.21.15.3:6806/2784566484","state":["exists","up"]},{"osd":2,"uuid":"8cf57c0d-5e18-448c-a419-e2d417a2d59b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":720,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":891966684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":891966684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6811","nonce":891966684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":891966684}]},"public_addr":"172.21.15.123:6813/891966684","cluster_addr":"172.21.15.123:6812/891966684","heartbeat_back_addr":"172.21.15.123:6811/891966684","heartbeat_front_addr":"172.21.15.123:6810/891966684","state":["exists","up"]},{"osd":3,"uuid":"15079b2b-ea92-49f2-8ab7-eb6edd581ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":720,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":4206675901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6805","nonce":4206675901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":4206675901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":4206675901}]},"public_addr":"172.21.15.123:6806/4206675901","cluster_addr":"172.21.15.123:6805/4206675901","heartbeat_back_addr":"172.21.15.123:6807/4206675901","heartbeat_front_addr":"172.21.15.123:6804/4206675901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]},{"pool":100,"snaps":[{"begin":1,"length":4}]},{"pool":101,"snaps":[{"begin":2,"length":4},{"begin":7,"length":2}]},{"pool":102,"snaps":[{"begin":1,"length":6}]},{"pool":103,"snaps":[{"begin":2,"length":9},{"begin":14,"length":1}]},{"pool":104,"snaps":[{"begin":2,"length":1}]},{"pool":179,"snaps":[{"begin":1,"length":1}]},{"pool":180,"snaps":[{"begin":1,"length":1}]},{"pool":182,"snaps":[{"begin":1,"length":1}]},{"pool":183,"snaps":[{"begin":1,"length":2}]},{"pool":184,"snaps":[{"begin":3,"length":1}]},{"pool":187,"snaps":[{"begin":2,"length":2}]},{"pool":189,"snaps":[{"begin":3,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T04:06:46.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.499+0000 7fbad547d640 1 -- 172.21.15.3:0/2758455375 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbabc03ead0 msgr2=0x7fbabc040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:46.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.499+0000 7fbad547d640 1 --2- 172.21.15.3:0/2758455375 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbabc03ead0 0x7fbabc040fe0 crc :-1 s=READY pgs=3872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:46.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.499+0000 7fbad547d640 1 -- 172.21.15.3:0/2758455375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad006e7b0 msgr2=0x7fbad01336a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:46.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.499+0000 7fbad547d640 1 --2- 172.21.15.3:0/2758455375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad006e7b0 0x7fbad01336a0 crc :-1 s=READY pgs=13013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:46.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.499+0000 7fbad547d640 1 -- 172.21.15.3:0/2758455375 shutdown_connections 2024-09-22T04:06:46.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.499+0000 7fbad547d640 1 --2- 172.21.15.3:0/2758455375 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fbabc03ead0 0x7fbabc040fe0 unknown :-1 s=CLOSED pgs=3872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:46.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.499+0000 7fbad547d640 1 --2- 172.21.15.3:0/2758455375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad006e7b0 0x7fbad01336a0 unknown :-1 s=CLOSED pgs=13013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:46.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.499+0000 7fbad547d640 1 -- 172.21.15.3:0/2758455375 >> 172.21.15.3:0/2758455375 conn(0x7fbad00a40f0 msgr2=0x7fbad0133c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:46.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.499+0000 7fbad547d640 1 -- 172.21.15.3:0/2758455375 shutdown_connections 2024-09-22T04:06:46.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:46.499+0000 7fbad547d640 1 -- 172.21.15.3:0/2758455375 wait complete. 2024-09-22T04:06:47.513 INFO:tasks.ceph:Scrubbing osd.0 2024-09-22T04:06:47.513 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-22T04:06:47.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.638+0000 7f4495391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.649+0000 7f4495391640 1 Processor -- start 2024-09-22T04:06:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.649+0000 7f4495391640 1 -- start start 2024-09-22T04:06:47.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.650+0000 7f4495391640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44900a3e10 0x7f44900a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:47.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.650+0000 7f4495391640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44900a5120 con 0x7f44900a4760 2024-09-22T04:06:47.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.650+0000 7f4495391640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f44900a5730 con 0x7f44900a3e10 2024-09-22T04:06:47.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.650+0000 7f448effd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44900a3e10 0x7f44900a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:47.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.650+0000 7f448e7fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f44900a4760 0x7f4490135e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42736/0 (socket says 172.21.15.3:42736) 2024-09-22T04:06:47.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.650+0000 7f448effd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f44900a3e10 0x7f44900a4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34020/0 (socket says 172.21.15.3:34020) 2024-09-22T04:06:47.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.650+0000 7f448e7fc640 1 -- 172.21.15.3:0/3314729543 learned_addr learned my addr 172.21.15.3:0/3314729543 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T04:06:47.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.651+0000 7f448effd640 1 -- 172.21.15.3:0/3314729543 >> v1:172.21.15.3:6789/0 conn(0x7f44900a4760 legacy=0x7f4490135e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:47.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.651+0000 7f448dffb640 1 -- 172.21.15.3:0/3314729543 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1659266881 0 0) 0x7f44900a5120 con 0x7f44900a4760 2024-09-22T04:06:47.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.651+0000 7f448effd640 1 -- 172.21.15.3:0/3314729543 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449006e770 con 0x7f44900a3e10 2024-09-22T04:06:47.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.651+0000 7f448effd640 1 --2- 172.21.15.3:0/3314729543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44900a3e10 0x7f44900a4220 crc :-1 s=READY pgs=13015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc61e2128afd9f45 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:47.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.651+0000 7f448dffb640 1 -- 172.21.15.3:0/3314729543 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f448000e040 con 0x7f44900a3e10 2024-09-22T04:06:47.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.651+0000 7f448dffb640 1 -- 172.21.15.3:0/3314729543 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4480007670 con 0x7f44900a3e10 2024-09-22T04:06:47.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.651+0000 7f448dffb640 1 -- 172.21.15.3:0/3314729543 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4480007910 con 0x7f44900a3e10 2024-09-22T04:06:47.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.652+0000 7f4495391640 1 -- 172.21.15.3:0/3314729543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44900a3e10 msgr2=0x7f44900a4220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:47.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.652+0000 7f4495391640 1 --2- 172.21.15.3:0/3314729543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44900a3e10 0x7f44900a4220 crc :-1 s=READY pgs=13015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:47.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.652+0000 7f4495391640 1 -- 172.21.15.3:0/3314729543 shutdown_connections 2024-09-22T04:06:47.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.652+0000 7f4495391640 1 --2- 172.21.15.3:0/3314729543 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44900a3e10 0x7f44900a4220 unknown :-1 s=CLOSED pgs=13015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:47.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.652+0000 7f4495391640 1 -- 172.21.15.3:0/3314729543 >> 172.21.15.3:0/3314729543 conn(0x7f44900a74f0 msgr2=0x7f44900a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:47.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.652+0000 7f4495391640 1 -- 172.21.15.3:0/3314729543 shutdown_connections 2024-09-22T04:06:47.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.652+0000 7f4495391640 1 -- 172.21.15.3:0/3314729543 wait complete. 2024-09-22T04:06:47.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.652+0000 7f4495391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:47.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.655+0000 7f4495391640 1 Processor -- start 2024-09-22T04:06:47.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.655+0000 7f4495391640 1 -- start start 2024-09-22T04:06:47.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.657+0000 7f4495391640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44900a3e10 0x7f4490124360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:47.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.657+0000 7f4495391640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4490123fa0 con 0x7f44900a3e10 2024-09-22T04:06:47.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.657+0000 7f448effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44900a3e10 0x7f4490124360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:47.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.657+0000 7f448effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44900a3e10 0x7f4490124360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34032/0 (socket says 172.21.15.3:34032) 2024-09-22T04:06:47.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.657+0000 7f448effd640 1 -- 172.21.15.3:0/607883191 learned_addr learned my addr 172.21.15.3:0/607883191 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:47.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.658+0000 7f448effd640 1 -- 172.21.15.3:0/607883191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4490125c20 con 0x7f44900a3e10 2024-09-22T04:06:47.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.658+0000 7f448effd640 1 --2- 172.21.15.3:0/607883191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44900a3e10 0x7f4490124360 crc :-1 s=READY pgs=13016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:47.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.658+0000 7f44877fe640 1 -- 172.21.15.3:0/607883191 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f448000e040 con 0x7f44900a3e10 2024-09-22T04:06:47.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.658+0000 7f44877fe640 1 -- 172.21.15.3:0/607883191 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4480013aa0 con 0x7f44900a3e10 2024-09-22T04:06:47.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.658+0000 7f4495391640 1 -- 172.21.15.3:0/607883191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4490126340 con 0x7f44900a3e10 2024-09-22T04:06:47.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.658+0000 7f44877fe640 1 -- 172.21.15.3:0/607883191 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4480013d40 con 0x7f44900a3e10 2024-09-22T04:06:47.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.658+0000 7f4495391640 1 -- 172.21.15.3:0/607883191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4490123c00 con 0x7f44900a3e10 2024-09-22T04:06:47.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.659+0000 7f4495391640 1 -- 172.21.15.3:0/607883191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f44900a43c0 con 0x7f44900a3e10 2024-09-22T04:06:47.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.659+0000 7f44877fe640 1 -- 172.21.15.3:0/607883191 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f4480011040 con 0x7f44900a3e10 2024-09-22T04:06:47.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.660+0000 7f44877fe640 1 --2- 172.21.15.3:0/607883191 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f447003eb00 0x7f4470041010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:47.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.660+0000 7f44877fe640 1 -- 172.21.15.3:0/607883191 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(733..733 src has 590..733) ==== 3431+0+0 (crc 0 0 0) 0x7f4480051500 con 0x7f44900a3e10 2024-09-22T04:06:47.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.660+0000 7f44877fe640 1 --2- 172.21.15.3:0/607883191 >> v2:172.21.15.3:6801/2889521405 conn(0x7f4470044c40 0x7f4470047130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:47.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.660+0000 7f44877fe640 1 -- 172.21.15.3:0/607883191 --> v2:172.21.15.3:6801/2889521405 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f44700477c0 con 0x7f4470044c40 2024-09-22T04:06:47.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.660+0000 7f44877fe640 1 -- 172.21.15.3:0/607883191 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=733) ==== 24+0+0 (crc 0 0 0) 0x7f44800539f0 con 0x7f44900a3e10 2024-09-22T04:06:47.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.660+0000 7f448e7fc640 1 --2- 172.21.15.3:0/607883191 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f447003eb00 0x7f4470041010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:47.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.660+0000 7f448e7fc640 1 --2- 172.21.15.3:0/607883191 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f447003eb00 0x7f4470041010 crc :-1 s=READY pgs=3873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:47.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.661+0000 7f448f7fe640 1 --2- 172.21.15.3:0/607883191 >> v2:172.21.15.3:6801/2889521405 conn(0x7f4470044c40 0x7f4470047130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:06:47.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.664+0000 7f448f7fe640 1 --2- 172.21.15.3:0/607883191 >> v2:172.21.15.3:6801/2889521405 conn(0x7f4470044c40 0x7f4470047130 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:47.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.666+0000 7f44877fe640 1 -- 172.21.15.3:0/607883191 <== osd.0 v2:172.21.15.3:6801/2889521405 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5790 (crc 0 0 0) 0x7f44700477c0 con 0x7f4470044c40 2024-09-22T04:06:47.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.669+0000 7f4495391640 1 -- 172.21.15.3:0/607883191 --> v2:172.21.15.3:6801/2889521405 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f44900a5c60 con 0x7f4470044c40 2024-09-22T04:06:47.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.671+0000 7f44877fe640 1 -- 172.21.15.3:0/607883191 <== osd.0 v2:172.21.15.3:6801/2889521405 2 ==== command_reply(tid 2: 0 ) ==== 8+0+22 (crc 0 0 0) 0x7f4488009830 con 0x7f4470044c40 2024-09-22T04:06:47.672 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-22T04:06:47.672 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "" 2024-09-22T04:06:47.673 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-22T04:06:47.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.671+0000 7f4495391640 1 -- 172.21.15.3:0/607883191 >> v2:172.21.15.3:6801/2889521405 conn(0x7f4470044c40 msgr2=0x7f4470047130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:47.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.671+0000 7f4495391640 1 --2- 172.21.15.3:0/607883191 >> v2:172.21.15.3:6801/2889521405 conn(0x7f4470044c40 0x7f4470047130 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:47.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.671+0000 7f4495391640 1 -- 172.21.15.3:0/607883191 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f447003eb00 msgr2=0x7f4470041010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:47.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.671+0000 7f4495391640 1 --2- 172.21.15.3:0/607883191 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f447003eb00 0x7f4470041010 crc :-1 s=READY pgs=3873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:47.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.672+0000 7f4495391640 1 -- 172.21.15.3:0/607883191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44900a3e10 msgr2=0x7f4490124360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:47.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.672+0000 7f4495391640 1 --2- 172.21.15.3:0/607883191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44900a3e10 0x7f4490124360 crc :-1 s=READY pgs=13016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:47.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.672+0000 7f4495391640 1 -- 172.21.15.3:0/607883191 shutdown_connections 2024-09-22T04:06:47.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.672+0000 7f4495391640 1 --2- 172.21.15.3:0/607883191 >> v2:172.21.15.3:6801/2889521405 conn(0x7f4470044c40 0x7f4470047130 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:47.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.672+0000 7f4495391640 1 --2- 172.21.15.3:0/607883191 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f447003eb00 0x7f4470041010 unknown :-1 s=CLOSED pgs=3873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:47.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.672+0000 7f4495391640 1 --2- 172.21.15.3:0/607883191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44900a3e10 0x7f4490124360 unknown :-1 s=CLOSED pgs=13016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:47.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.672+0000 7f4495391640 1 -- 172.21.15.3:0/607883191 >> 172.21.15.3:0/607883191 conn(0x7f44900a74f0 msgr2=0x7f4490071390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:47.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.672+0000 7f4495391640 1 -- 172.21.15.3:0/607883191 shutdown_connections 2024-09-22T04:06:47.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.672+0000 7f4495391640 1 -- 172.21.15.3:0/607883191 wait complete. 2024-09-22T04:06:47.684 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-22T04:06:47.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.792+0000 7f4aaed11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:47.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.799+0000 7f4aaed11640 1 Processor -- start 2024-09-22T04:06:47.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.799+0000 7f4aaed11640 1 -- start start 2024-09-22T04:06:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.800+0000 7f4aaed11640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4aa8136070 0x7f4aa8136480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.800+0000 7f4aaed11640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aa809efe0 con 0x7f4aa80a04f0 2024-09-22T04:06:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.800+0000 7f4aaed11640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4aa8135c20 con 0x7f4aa8136070 2024-09-22T04:06:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.800+0000 7f4aacaad640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4aa8136070 0x7f4aa8136480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:47.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.800+0000 7f4aacaad640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4aa8136070 0x7f4aa8136480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34040/0 (socket says 172.21.15.3:34040) 2024-09-22T04:06:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.800+0000 7f4aacaad640 1 -- 172.21.15.3:0/1233154383 learned_addr learned my addr 172.21.15.3:0/1233154383 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.801+0000 7f4aacaad640 1 -- 172.21.15.3:0/1233154383 >> v1:172.21.15.3:6789/0 conn(0x7f4aa80a04f0 legacy=0x7f4aa80a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.801+0000 7f4aa77fe640 1 -- 172.21.15.3:0/1233154383 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4020719683 0 0) 0x7f4aa809efe0 con 0x7f4aa80a04f0 2024-09-22T04:06:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.801+0000 7f4aacaad640 1 -- 172.21.15.3:0/1233154383 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa809d360 con 0x7f4aa8136070 2024-09-22T04:06:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.801+0000 7f4aacaad640 1 --2- 172.21.15.3:0/1233154383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa8136070 0x7f4aa8136480 crc :-1 s=READY pgs=13018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228ecee01125137f server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.801+0000 7f4aa77fe640 1 -- 172.21.15.3:0/1233154383 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4a98009e40 con 0x7f4aa8136070 2024-09-22T04:06:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.801+0000 7f4aa77fe640 1 -- 172.21.15.3:0/1233154383 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4a9800e230 con 0x7f4aa8136070 2024-09-22T04:06:47.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.801+0000 7f4aa77fe640 1 -- 172.21.15.3:0/1233154383 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4a9800e4d0 con 0x7f4aa8136070 2024-09-22T04:06:47.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.802+0000 7f4aaed11640 1 -- 172.21.15.3:0/1233154383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa8136070 msgr2=0x7f4aa8136480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:47.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.802+0000 7f4aaed11640 1 --2- 172.21.15.3:0/1233154383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa8136070 0x7f4aa8136480 crc :-1 s=READY pgs=13018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:47.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.802+0000 7f4aaed11640 1 -- 172.21.15.3:0/1233154383 shutdown_connections 2024-09-22T04:06:47.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.802+0000 7f4aaed11640 1 --2- 172.21.15.3:0/1233154383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa8136070 0x7f4aa8136480 unknown :-1 s=CLOSED pgs=13018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:47.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.802+0000 7f4aaed11640 1 -- 172.21.15.3:0/1233154383 >> 172.21.15.3:0/1233154383 conn(0x7f4aa806eb10 msgr2=0x7f4aa806ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:47.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.802+0000 7f4aaed11640 1 -- 172.21.15.3:0/1233154383 shutdown_connections 2024-09-22T04:06:47.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.802+0000 7f4aaed11640 1 -- 172.21.15.3:0/1233154383 wait complete. 2024-09-22T04:06:47.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.802+0000 7f4aaed11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:47.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.804+0000 7f4aaed11640 1 Processor -- start 2024-09-22T04:06:47.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.804+0000 7f4aaed11640 1 -- start start 2024-09-22T04:06:47.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.805+0000 7f4aaed11640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa80a04f0 0x7f4aa8122a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:47.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.805+0000 7f4aaed11640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa8126960 con 0x7f4aa80a04f0 2024-09-22T04:06:47.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.806+0000 7f4aacaad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa80a04f0 0x7f4aa8122a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.806+0000 7f4aacaad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa80a04f0 0x7f4aa8122a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34048/0 (socket says 172.21.15.3:34048) 2024-09-22T04:06:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.806+0000 7f4aacaad640 1 -- 172.21.15.3:0/438617937 learned_addr learned my addr 172.21.15.3:0/438617937 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.806+0000 7f4aacaad640 1 -- 172.21.15.3:0/438617937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa80a3f90 con 0x7f4aa80a04f0 2024-09-22T04:06:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.806+0000 7f4aacaad640 1 --2- 172.21.15.3:0/438617937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa80a04f0 0x7f4aa8122a20 crc :-1 s=READY pgs=13019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.806+0000 7f4aa57fa640 1 -- 172.21.15.3:0/438617937 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4a9800edf0 con 0x7f4aa80a04f0 2024-09-22T04:06:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.806+0000 7f4aa57fa640 1 -- 172.21.15.3:0/438617937 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f4a98015420 con 0x7f4aa80a04f0 2024-09-22T04:06:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.806+0000 7f4aaed11640 1 -- 172.21.15.3:0/438617937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa8125d00 con 0x7f4aa80a04f0 2024-09-22T04:06:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.806+0000 7f4aa57fa640 1 -- 172.21.15.3:0/438617937 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f4a980156c0 con 0x7f4aa80a04f0 2024-09-22T04:06:47.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.806+0000 7f4aaed11640 1 -- 172.21.15.3:0/438617937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa81259f0 con 0x7f4aa80a04f0 2024-09-22T04:06:47.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.807+0000 7f4aaed11640 1 -- 172.21.15.3:0/438617937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aa806a460 con 0x7f4aa80a04f0 2024-09-22T04:06:47.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.807+0000 7f4aa57fa640 1 -- 172.21.15.3:0/438617937 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f4a98015870 con 0x7f4aa80a04f0 2024-09-22T04:06:47.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.808+0000 7f4aa57fa640 1 --2- 172.21.15.3:0/438617937 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4a8803ead0 0x7f4a88040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:47.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.808+0000 7f4aa57fa640 1 -- 172.21.15.3:0/438617937 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(733..733 src has 590..733) ==== 3431+0+0 (crc 0 0 0) 0x7f4a98050fc0 con 0x7f4aa80a04f0 2024-09-22T04:06:47.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.808+0000 7f4aa7fff640 1 --2- 172.21.15.3:0/438617937 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4a8803ead0 0x7f4a88040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:47.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.809+0000 7f4aa7fff640 1 --2- 172.21.15.3:0/438617937 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4a8803ead0 0x7f4a88040fe0 crc :-1 s=READY pgs=3874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:47.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:47.813+0000 7f4aa57fa640 1 -- 172.21.15.3:0/438617937 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f4a98014c60 con 0x7f4aa80a04f0 2024-09-22T04:06:48.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.001+0000 7f4aaed11640 1 -- 172.21.15.3:0/438617937 --> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f4aa809d680 con 0x7f4a8803ead0 2024-09-22T04:06:48.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.002+0000 7f4aa57fa640 1 -- 172.21.15.3:0/438617937 <== mgr.4099 v2:172.21.15.123:6800/551784226 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (crc 0 0 0) 0x7f4a900012b0 con 0x7f4a8803ead0 2024-09-22T04:06:48.003 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 0 to deep-scrub 2024-09-22T04:06:48.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.004+0000 7f4aaed11640 1 -- 172.21.15.3:0/438617937 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4a8803ead0 msgr2=0x7f4a88040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.004+0000 7f4aaed11640 1 --2- 172.21.15.3:0/438617937 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4a8803ead0 0x7f4a88040fe0 crc :-1 s=READY pgs=3874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.004+0000 7f4aaed11640 1 -- 172.21.15.3:0/438617937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa80a04f0 msgr2=0x7f4aa8122a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.004+0000 7f4aaed11640 1 --2- 172.21.15.3:0/438617937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa80a04f0 0x7f4aa8122a20 crc :-1 s=READY pgs=13019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.005+0000 7f4aaed11640 1 -- 172.21.15.3:0/438617937 shutdown_connections 2024-09-22T04:06:48.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.005+0000 7f4aaed11640 1 --2- 172.21.15.3:0/438617937 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f4a8803ead0 0x7f4a88040fe0 unknown :-1 s=CLOSED pgs=3874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.005+0000 7f4aaed11640 1 --2- 172.21.15.3:0/438617937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4aa80a04f0 0x7f4aa8122a20 unknown :-1 s=CLOSED pgs=13019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.005+0000 7f4aaed11640 1 -- 172.21.15.3:0/438617937 >> 172.21.15.3:0/438617937 conn(0x7f4aa806eb10 msgr2=0x7f4aa80a4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:48.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.005+0000 7f4aaed11640 1 -- 172.21.15.3:0/438617937 shutdown_connections 2024-09-22T04:06:48.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.005+0000 7f4aaed11640 1 -- 172.21.15.3:0/438617937 wait complete. 2024-09-22T04:06:48.026 INFO:tasks.ceph:Scrubbing osd.1 2024-09-22T04:06:48.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-22T04:06:48.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.146+0000 7fb483b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:48.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.155+0000 7fb483b91640 1 Processor -- start 2024-09-22T04:06:48.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.156+0000 7fb483b91640 1 -- start start 2024-09-22T04:06:48.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.157+0000 7fb483b91640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb47c09dca0 0x7fb47c06e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:48.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.157+0000 7fb483b91640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb47c0a4dc0 con 0x7fb47c06ecf0 2024-09-22T04:06:48.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.157+0000 7fb483b91640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb47c0a4ab0 con 0x7fb47c09dca0 2024-09-22T04:06:48.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.157+0000 7fb48192d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb47c09dca0 0x7fb47c06e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:48.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.157+0000 7fb48192d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb47c09dca0 0x7fb47c06e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34062/0 (socket says 172.21.15.3:34062) 2024-09-22T04:06:48.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.157+0000 7fb48112c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb47c06ecf0 0x7fb47c133a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42756/0 (socket says 172.21.15.3:42756) 2024-09-22T04:06:48.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.157+0000 7fb48192d640 1 -- 172.21.15.3:0/4142791916 learned_addr learned my addr 172.21.15.3:0/4142791916 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:48.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.158+0000 7fb48192d640 1 -- 172.21.15.3:0/4142791916 >> v1:172.21.15.3:6789/0 conn(0x7fb47c06ecf0 legacy=0x7fb47c133a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.158+0000 7fb48192d640 1 -- 172.21.15.3:0/4142791916 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c09ef50 con 0x7fb47c09dca0 2024-09-22T04:06:48.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.158+0000 7fb48092b640 1 -- 172.21.15.3:0/4142791916 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2337390451 0 0) 0x7fb47c0a4dc0 con 0x7fb47c06ecf0 2024-09-22T04:06:48.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.158+0000 7fb48192d640 1 --2- 172.21.15.3:0/4142791916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb47c09dca0 0x7fb47c06e7b0 crc :-1 s=READY pgs=13021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474211ace825f2b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:48.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.158+0000 7fb48092b640 1 -- 172.21.15.3:0/4142791916 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb478009e50 con 0x7fb47c09dca0 2024-09-22T04:06:48.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.158+0000 7fb48092b640 1 -- 172.21.15.3:0/4142791916 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb47800e230 con 0x7fb47c09dca0 2024-09-22T04:06:48.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.158+0000 7fb48092b640 1 -- 172.21.15.3:0/4142791916 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb47800e4d0 con 0x7fb47c09dca0 2024-09-22T04:06:48.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.159+0000 7fb483b91640 1 -- 172.21.15.3:0/4142791916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb47c09dca0 msgr2=0x7fb47c06e7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.159+0000 7fb483b91640 1 --2- 172.21.15.3:0/4142791916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb47c09dca0 0x7fb47c06e7b0 crc :-1 s=READY pgs=13021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.159+0000 7fb483b91640 1 -- 172.21.15.3:0/4142791916 shutdown_connections 2024-09-22T04:06:48.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.159+0000 7fb483b91640 1 --2- 172.21.15.3:0/4142791916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb47c09dca0 0x7fb47c06e7b0 unknown :-1 s=CLOSED pgs=13021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.159+0000 7fb483b91640 1 -- 172.21.15.3:0/4142791916 >> 172.21.15.3:0/4142791916 conn(0x7fb47c0a39b0 msgr2=0x7fb47c0a3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:48.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.159+0000 7fb483b91640 1 -- 172.21.15.3:0/4142791916 shutdown_connections 2024-09-22T04:06:48.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.159+0000 7fb483b91640 1 -- 172.21.15.3:0/4142791916 wait complete. 2024-09-22T04:06:48.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.159+0000 7fb483b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:48.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.161+0000 7fb483b91640 1 Processor -- start 2024-09-22T04:06:48.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.161+0000 7fb483b91640 1 -- start start 2024-09-22T04:06:48.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.162+0000 7fb483b91640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb47c06ecf0 0x7fb47c0728e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:48.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.162+0000 7fb483b91640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c126cb0 con 0x7fb47c06ecf0 2024-09-22T04:06:48.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.162+0000 7fb48192d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb47c06ecf0 0x7fb47c0728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:48.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.163+0000 7fb48192d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb47c06ecf0 0x7fb47c0728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34070/0 (socket says 172.21.15.3:34070) 2024-09-22T04:06:48.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.163+0000 7fb48192d640 1 -- 172.21.15.3:0/3007224006 learned_addr learned my addr 172.21.15.3:0/3007224006 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.163+0000 7fb48192d640 1 -- 172.21.15.3:0/3007224006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c1344e0 con 0x7fb47c06ecf0 2024-09-22T04:06:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.163+0000 7fb48192d640 1 --2- 172.21.15.3:0/3007224006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb47c06ecf0 0x7fb47c0728e0 crc :-1 s=READY pgs=13022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.163+0000 7fb4727fc640 1 -- 172.21.15.3:0/3007224006 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb47800ee50 con 0x7fb47c06ecf0 2024-09-22T04:06:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.163+0000 7fb4727fc640 1 -- 172.21.15.3:0/3007224006 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fb478015420 con 0x7fb47c06ecf0 2024-09-22T04:06:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.163+0000 7fb483b91640 1 -- 172.21.15.3:0/3007224006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47c074350 con 0x7fb47c06ecf0 2024-09-22T04:06:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.163+0000 7fb4727fc640 1 -- 172.21.15.3:0/3007224006 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fb4780156c0 con 0x7fb47c06ecf0 2024-09-22T04:06:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.163+0000 7fb483b91640 1 -- 172.21.15.3:0/3007224006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47c127650 con 0x7fb47c06ecf0 2024-09-22T04:06:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.164+0000 7fb4727fc640 1 -- 172.21.15.3:0/3007224006 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fb478015870 con 0x7fb47c06ecf0 2024-09-22T04:06:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.164+0000 7fb483b91640 1 -- 172.21.15.3:0/3007224006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb47c09e3e0 con 0x7fb47c06ecf0 2024-09-22T04:06:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.164+0000 7fb4727fc640 1 --2- 172.21.15.3:0/3007224006 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fb46003eb20 0x7fb460041030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.165+0000 7fb4727fc640 1 -- 172.21.15.3:0/3007224006 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(733..733 src has 590..733) ==== 3431+0+0 (crc 0 0 0) 0x7fb478051200 con 0x7fb47c06ecf0 2024-09-22T04:06:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.165+0000 7fb4727fc640 1 --2- 172.21.15.3:0/3007224006 >> v2:172.21.15.3:6802/2784566484 conn(0x7fb460044c60 0x7fb460047150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.165+0000 7fb4727fc640 1 -- 172.21.15.3:0/3007224006 --> v2:172.21.15.3:6802/2784566484 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4600477e0 con 0x7fb460044c60 2024-09-22T04:06:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.165+0000 7fb4727fc640 1 -- 172.21.15.3:0/3007224006 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=733) ==== 24+0+0 (crc 0 0 0) 0x7fb478051540 con 0x7fb47c06ecf0 2024-09-22T04:06:48.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.165+0000 7fb48112c640 1 --2- 172.21.15.3:0/3007224006 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fb46003eb20 0x7fb460041030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:48.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.165+0000 7fb48112c640 1 --2- 172.21.15.3:0/3007224006 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fb46003eb20 0x7fb460041030 crc :-1 s=READY pgs=3875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:48.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.166+0000 7fb48212e640 1 --2- 172.21.15.3:0/3007224006 >> v2:172.21.15.3:6802/2784566484 conn(0x7fb460044c60 0x7fb460047150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:06:48.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.169+0000 7fb48212e640 1 --2- 172.21.15.3:0/3007224006 >> v2:172.21.15.3:6802/2784566484 conn(0x7fb460044c60 0x7fb460047150 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:48.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.171+0000 7fb4727fc640 1 -- 172.21.15.3:0/3007224006 <== osd.1 v2:172.21.15.3:6802/2784566484 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5790 (crc 0 0 0) 0x7fb4600477e0 con 0x7fb460044c60 2024-09-22T04:06:48.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.174+0000 7fb483b91640 1 -- 172.21.15.3:0/3007224006 --> v2:172.21.15.3:6802/2784566484 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb47c06e7b0 con 0x7fb460044c60 2024-09-22T04:06:48.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.176+0000 7fb4727fc640 1 -- 172.21.15.3:0/3007224006 <== osd.1 v2:172.21.15.3:6802/2784566484 2 ==== command_reply(tid 2: 0 ) ==== 8+0+22 (crc 0 0 0) 0x7fb47c06e7b0 con 0x7fb460044c60 2024-09-22T04:06:48.177 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-22T04:06:48.177 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "" 2024-09-22T04:06:48.177 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-22T04:06:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.176+0000 7fb483b91640 1 -- 172.21.15.3:0/3007224006 >> v2:172.21.15.3:6802/2784566484 conn(0x7fb460044c60 msgr2=0x7fb460047150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.176+0000 7fb483b91640 1 --2- 172.21.15.3:0/3007224006 >> v2:172.21.15.3:6802/2784566484 conn(0x7fb460044c60 0x7fb460047150 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.176+0000 7fb483b91640 1 -- 172.21.15.3:0/3007224006 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fb46003eb20 msgr2=0x7fb460041030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.176+0000 7fb483b91640 1 --2- 172.21.15.3:0/3007224006 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fb46003eb20 0x7fb460041030 crc :-1 s=READY pgs=3875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.176+0000 7fb483b91640 1 -- 172.21.15.3:0/3007224006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb47c06ecf0 msgr2=0x7fb47c0728e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.176+0000 7fb483b91640 1 --2- 172.21.15.3:0/3007224006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb47c06ecf0 0x7fb47c0728e0 crc :-1 s=READY pgs=13022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.176+0000 7fb483b91640 1 -- 172.21.15.3:0/3007224006 shutdown_connections 2024-09-22T04:06:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.176+0000 7fb483b91640 1 --2- 172.21.15.3:0/3007224006 >> v2:172.21.15.3:6802/2784566484 conn(0x7fb460044c60 0x7fb460047150 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.176+0000 7fb483b91640 1 --2- 172.21.15.3:0/3007224006 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fb46003eb20 0x7fb460041030 unknown :-1 s=CLOSED pgs=3875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.176+0000 7fb483b91640 1 --2- 172.21.15.3:0/3007224006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb47c06ecf0 0x7fb47c0728e0 unknown :-1 s=CLOSED pgs=13022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.176+0000 7fb483b91640 1 -- 172.21.15.3:0/3007224006 >> 172.21.15.3:0/3007224006 conn(0x7fb47c0a39b0 msgr2=0x7fb47c070750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:48.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.176+0000 7fb483b91640 1 -- 172.21.15.3:0/3007224006 shutdown_connections 2024-09-22T04:06:48.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.176+0000 7fb483b91640 1 -- 172.21.15.3:0/3007224006 wait complete. 2024-09-22T04:06:48.188 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-22T04:06:48.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.296+0000 7f593e991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:48.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.303+0000 7f593e991640 1 Processor -- start 2024-09-22T04:06:48.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.303+0000 7f593e991640 1 -- start start 2024-09-22T04:06:48.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.304+0000 7f593e991640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f593806f2d0 0x7f593813ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:48.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.304+0000 7f593e991640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59380a11d0 con 0x7f593806e7b0 2024-09-22T04:06:48.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.304+0000 7f593e991640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f59380a2a20 con 0x7f593806f2d0 2024-09-22T04:06:48.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.304+0000 7f59377fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f593806f2d0 0x7f593813ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:48.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.304+0000 7f59377fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f593806f2d0 0x7f593813ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34084/0 (socket says 172.21.15.3:34084) 2024-09-22T04:06:48.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.304+0000 7f5937fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f593806e7b0 0x7f593806ebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42768/0 (socket says 172.21.15.3:42768) 2024-09-22T04:06:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.304+0000 7f59377fe640 1 -- 172.21.15.3:0/889925109 learned_addr learned my addr 172.21.15.3:0/889925109 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.305+0000 7f59377fe640 1 -- 172.21.15.3:0/889925109 >> v1:172.21.15.3:6789/0 conn(0x7f593806e7b0 legacy=0x7f593806ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.305+0000 7f59377fe640 1 -- 172.21.15.3:0/889925109 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59380a0df0 con 0x7f593806f2d0 2024-09-22T04:06:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.305+0000 7f59377fe640 1 --2- 172.21.15.3:0/889925109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593806f2d0 0x7f593813ae40 crc :-1 s=READY pgs=13024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e81603e65e3267 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.305+0000 7f5936ffd640 1 -- 172.21.15.3:0/889925109 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f592c009e50 con 0x7f593806f2d0 2024-09-22T04:06:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.305+0000 7f5936ffd640 1 -- 172.21.15.3:0/889925109 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f592c00e230 con 0x7f593806f2d0 2024-09-22T04:06:48.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.305+0000 7f5936ffd640 1 -- 172.21.15.3:0/889925109 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f592c00e4d0 con 0x7f593806f2d0 2024-09-22T04:06:48.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.306+0000 7f593e991640 1 -- 172.21.15.3:0/889925109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593806f2d0 msgr2=0x7f593813ae40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.306+0000 7f593e991640 1 --2- 172.21.15.3:0/889925109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593806f2d0 0x7f593813ae40 crc :-1 s=READY pgs=13024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.306+0000 7f593e991640 1 -- 172.21.15.3:0/889925109 shutdown_connections 2024-09-22T04:06:48.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.306+0000 7f593e991640 1 --2- 172.21.15.3:0/889925109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593806f2d0 0x7f593813ae40 unknown :-1 s=CLOSED pgs=13024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.306+0000 7f593e991640 1 -- 172.21.15.3:0/889925109 >> 172.21.15.3:0/889925109 conn(0x7f59380a40f0 msgr2=0x7f59380a4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:48.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.306+0000 7f593e991640 1 -- 172.21.15.3:0/889925109 shutdown_connections 2024-09-22T04:06:48.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.306+0000 7f593e991640 1 -- 172.21.15.3:0/889925109 wait complete. 2024-09-22T04:06:48.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.306+0000 7f593e991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:48.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.308+0000 7f593e991640 1 Processor -- start 2024-09-22T04:06:48.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.308+0000 7f593e991640 1 -- start start 2024-09-22T04:06:48.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.310+0000 7f593e991640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593806e7b0 0x7f593811e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.310+0000 7f593e991640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f593811f960 con 0x7f593806e7b0 2024-09-22T04:06:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.310+0000 7f5937fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593806e7b0 0x7f593811e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.310+0000 7f5937fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593806e7b0 0x7f593811e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34090/0 (socket says 172.21.15.3:34090) 2024-09-22T04:06:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.310+0000 7f5937fff640 1 -- 172.21.15.3:0/915166928 learned_addr learned my addr 172.21.15.3:0/915166928 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.310+0000 7f5937fff640 1 -- 172.21.15.3:0/915166928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59381338b0 con 0x7f593806e7b0 2024-09-22T04:06:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.310+0000 7f5937fff640 1 --2- 172.21.15.3:0/915166928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593806e7b0 0x7f593811e620 crc :-1 s=READY pgs=13025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.310+0000 7f5934ff9640 1 -- 172.21.15.3:0/915166928 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f59280059a0 con 0x7f593806e7b0 2024-09-22T04:06:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.310+0000 7f5934ff9640 1 -- 172.21.15.3:0/915166928 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f5928005b00 con 0x7f593806e7b0 2024-09-22T04:06:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.310+0000 7f5934ff9640 1 -- 172.21.15.3:0/915166928 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5928005da0 con 0x7f593806e7b0 2024-09-22T04:06:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.310+0000 7f593e991640 1 -- 172.21.15.3:0/915166928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5938122600 con 0x7f593806e7b0 2024-09-22T04:06:48.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.310+0000 7f593e991640 1 -- 172.21.15.3:0/915166928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5938133ab0 con 0x7f593806e7b0 2024-09-22T04:06:48.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.311+0000 7f5934ff9640 1 -- 172.21.15.3:0/915166928 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f59280070b0 con 0x7f593806e7b0 2024-09-22T04:06:48.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.311+0000 7f593e991640 1 -- 172.21.15.3:0/915166928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59381208d0 con 0x7f593806e7b0 2024-09-22T04:06:48.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.312+0000 7f5934ff9640 1 --2- 172.21.15.3:0/915166928 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f592403ead0 0x7f5924040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:48.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.312+0000 7f5934ff9640 1 -- 172.21.15.3:0/915166928 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(733..733 src has 590..733) ==== 3431+0+0 (crc 0 0 0) 0x7f592804fb40 con 0x7f593806e7b0 2024-09-22T04:06:48.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.312+0000 7f59377fe640 1 --2- 172.21.15.3:0/915166928 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f592403ead0 0x7f5924040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:48.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.313+0000 7f59377fe640 1 --2- 172.21.15.3:0/915166928 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f592403ead0 0x7f5924040fe0 crc :-1 s=READY pgs=3876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:48.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.317+0000 7f5934ff9640 1 -- 172.21.15.3:0/915166928 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f592800f230 con 0x7f593806e7b0 2024-09-22T04:06:48.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.504+0000 7f593e991640 1 -- 172.21.15.3:0/915166928 --> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f5938074310 con 0x7f592403ead0 2024-09-22T04:06:48.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.505+0000 7f5934ff9640 1 -- 172.21.15.3:0/915166928 <== mgr.4099 v2:172.21.15.123:6800/551784226 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (crc 0 0 0) 0x7f592c00e4c0 con 0x7f592403ead0 2024-09-22T04:06:48.506 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 1 to deep-scrub 2024-09-22T04:06:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.508+0000 7f593e991640 1 -- 172.21.15.3:0/915166928 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f592403ead0 msgr2=0x7f5924040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.508+0000 7f593e991640 1 --2- 172.21.15.3:0/915166928 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f592403ead0 0x7f5924040fe0 crc :-1 s=READY pgs=3876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.508+0000 7f593e991640 1 -- 172.21.15.3:0/915166928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593806e7b0 msgr2=0x7f593811e620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.508+0000 7f593e991640 1 --2- 172.21.15.3:0/915166928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593806e7b0 0x7f593811e620 crc :-1 s=READY pgs=13025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.508+0000 7f593e991640 1 -- 172.21.15.3:0/915166928 shutdown_connections 2024-09-22T04:06:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.508+0000 7f593e991640 1 --2- 172.21.15.3:0/915166928 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f592403ead0 0x7f5924040fe0 unknown :-1 s=CLOSED pgs=3876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.508+0000 7f593e991640 1 --2- 172.21.15.3:0/915166928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f593806e7b0 0x7f593811e620 unknown :-1 s=CLOSED pgs=13025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.508+0000 7f593e991640 1 -- 172.21.15.3:0/915166928 >> 172.21.15.3:0/915166928 conn(0x7f59380a40f0 msgr2=0x7f5938133cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.509+0000 7f593e991640 1 -- 172.21.15.3:0/915166928 shutdown_connections 2024-09-22T04:06:48.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.509+0000 7f593e991640 1 -- 172.21.15.3:0/915166928 wait complete. 2024-09-22T04:06:48.529 INFO:tasks.ceph:Scrubbing osd.2 2024-09-22T04:06:48.529 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-22T04:06:48.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.644+0000 7fa71867f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:48.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.653+0000 7fa71867f640 1 Processor -- start 2024-09-22T04:06:48.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.654+0000 7fa71867f640 1 -- start start 2024-09-22T04:06:48.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.655+0000 7fa71867f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa71006eec0 0x7fa710138e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:48.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.655+0000 7fa71867f640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7100a4dc0 con 0x7fa71009dca0 2024-09-22T04:06:48.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.655+0000 7fa71867f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa7100a4ab0 con 0x7fa71006eec0 2024-09-22T04:06:48.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.655+0000 7fa715c1a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa71006eec0 0x7fa710138e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:48.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.655+0000 7fa715c1a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa71006eec0 0x7fa710138e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34092/0 (socket says 172.21.15.3:34092) 2024-09-22T04:06:48.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.655+0000 7fa71641b640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa71009dca0 0x7fa71006e7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42784/0 (socket says 172.21.15.3:42784) 2024-09-22T04:06:48.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.655+0000 7fa715c1a640 1 -- 172.21.15.3:0/1212106427 learned_addr learned my addr 172.21.15.3:0/1212106427 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:48.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.656+0000 7fa715c1a640 1 -- 172.21.15.3:0/1212106427 >> v1:172.21.15.3:6789/0 conn(0x7fa71009dca0 legacy=0x7fa71006e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.656+0000 7fa715c1a640 1 -- 172.21.15.3:0/1212106427 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71009ef50 con 0x7fa71006eec0 2024-09-22T04:06:48.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.656+0000 7fa715419640 1 -- 172.21.15.3:0/1212106427 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1423175484 0 0) 0x7fa7100a4dc0 con 0x7fa71009dca0 2024-09-22T04:06:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.656+0000 7fa715c1a640 1 --2- 172.21.15.3:0/1212106427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006eec0 0x7fa710138e70 crc :-1 s=READY pgs=13027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7613db2c4b13809 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.656+0000 7fa715419640 1 -- 172.21.15.3:0/1212106427 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa700009e50 con 0x7fa71006eec0 2024-09-22T04:06:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.656+0000 7fa715419640 1 -- 172.21.15.3:0/1212106427 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa70000e230 con 0x7fa71006eec0 2024-09-22T04:06:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.656+0000 7fa715419640 1 -- 172.21.15.3:0/1212106427 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa70000e4d0 con 0x7fa71006eec0 2024-09-22T04:06:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.657+0000 7fa71867f640 1 -- 172.21.15.3:0/1212106427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006eec0 msgr2=0x7fa710138e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.657+0000 7fa71867f640 1 --2- 172.21.15.3:0/1212106427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006eec0 0x7fa710138e70 crc :-1 s=READY pgs=13027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.657+0000 7fa71867f640 1 -- 172.21.15.3:0/1212106427 shutdown_connections 2024-09-22T04:06:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.657+0000 7fa71867f640 1 --2- 172.21.15.3:0/1212106427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006eec0 0x7fa710138e70 unknown :-1 s=CLOSED pgs=13027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.657+0000 7fa71867f640 1 -- 172.21.15.3:0/1212106427 >> 172.21.15.3:0/1212106427 conn(0x7fa7100a39b0 msgr2=0x7fa7100a3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.657+0000 7fa71867f640 1 -- 172.21.15.3:0/1212106427 shutdown_connections 2024-09-22T04:06:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.657+0000 7fa71867f640 1 -- 172.21.15.3:0/1212106427 wait complete. 2024-09-22T04:06:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.657+0000 7fa71867f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:48.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.659+0000 7fa71867f640 1 Processor -- start 2024-09-22T04:06:48.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.659+0000 7fa71867f640 1 -- start start 2024-09-22T04:06:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.660+0000 7fa71867f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006eec0 0x7fa710072990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.660+0000 7fa71867f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa710125c40 con 0x7fa71006eec0 2024-09-22T04:06:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.660+0000 7fa71641b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006eec0 0x7fa710072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.660+0000 7fa71641b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006eec0 0x7fa710072990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34094/0 (socket says 172.21.15.3:34094) 2024-09-22T04:06:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.661+0000 7fa71641b640 1 -- 172.21.15.3:0/1614805270 learned_addr learned my addr 172.21.15.3:0/1614805270 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.661+0000 7fa71641b640 1 -- 172.21.15.3:0/1614805270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa710125eb0 con 0x7fa71006eec0 2024-09-22T04:06:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.661+0000 7fa71641b640 1 --2- 172.21.15.3:0/1614805270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006eec0 0x7fa710072990 crc :-1 s=READY pgs=13028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.661+0000 7fa706ffd640 1 -- 172.21.15.3:0/1614805270 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa70c0070b0 con 0x7fa71006eec0 2024-09-22T04:06:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.661+0000 7fa706ffd640 1 -- 172.21.15.3:0/1614805270 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fa70c007210 con 0x7fa71006eec0 2024-09-22T04:06:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.661+0000 7fa71867f640 1 -- 172.21.15.3:0/1614805270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa710073c40 con 0x7fa71006eec0 2024-09-22T04:06:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.661+0000 7fa706ffd640 1 -- 172.21.15.3:0/1614805270 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fa70c0074b0 con 0x7fa71006eec0 2024-09-22T04:06:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.661+0000 7fa71867f640 1 -- 172.21.15.3:0/1614805270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7101273d0 con 0x7fa71006eec0 2024-09-22T04:06:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.662+0000 7fa71867f640 1 -- 172.21.15.3:0/1614805270 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa710127940 con 0x7fa71006eec0 2024-09-22T04:06:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.662+0000 7fa706ffd640 1 -- 172.21.15.3:0/1614805270 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fa70c007660 con 0x7fa71006eec0 2024-09-22T04:06:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.662+0000 7fa706ffd640 1 --2- 172.21.15.3:0/1614805270 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa6f403ead0 0x7fa6f4040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.662+0000 7fa706ffd640 1 -- 172.21.15.3:0/1614805270 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(733..733 src has 602..733) ==== 3431+0+0 (crc 0 0 0) 0x7fa70c050730 con 0x7fa71006eec0 2024-09-22T04:06:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.663+0000 7fa706ffd640 1 --2- 172.21.15.3:0/1614805270 >> v2:172.21.15.123:6813/891966684 conn(0x7fa6f4044c10 0x7fa6f4047100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.663+0000 7fa706ffd640 1 -- 172.21.15.3:0/1614805270 --> v2:172.21.15.123:6813/891966684 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa6f4047790 con 0x7fa6f4044c10 2024-09-22T04:06:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.663+0000 7fa706ffd640 1 -- 172.21.15.3:0/1614805270 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=733) ==== 24+0+0 (crc 0 0 0) 0x7fa70c052c20 con 0x7fa71006eec0 2024-09-22T04:06:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.663+0000 7fa715c1a640 1 --2- 172.21.15.3:0/1614805270 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa6f403ead0 0x7fa6f4040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.663+0000 7fa715c1a640 1 --2- 172.21.15.3:0/1614805270 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa6f403ead0 0x7fa6f4040fe0 crc :-1 s=READY pgs=3877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.663+0000 7fa716c1c640 1 --2- 172.21.15.3:0/1614805270 >> v2:172.21.15.123:6813/891966684 conn(0x7fa6f4044c10 0x7fa6f4047100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:06:48.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.667+0000 7fa716c1c640 1 --2- 172.21.15.3:0/1614805270 >> v2:172.21.15.123:6813/891966684 conn(0x7fa6f4044c10 0x7fa6f4047100 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.668+0000 7fa706ffd640 1 -- 172.21.15.3:0/1614805270 <== osd.2 v2:172.21.15.123:6813/891966684 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5790 (crc 0 0 0) 0x7fa6f4047790 con 0x7fa6f4044c10 2024-09-22T04:06:48.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.672+0000 7fa71867f640 1 -- 172.21.15.3:0/1614805270 --> v2:172.21.15.123:6813/891966684 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa71009e3e0 con 0x7fa6f4044c10 2024-09-22T04:06:48.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.673+0000 7fa706ffd640 1 -- 172.21.15.3:0/1614805270 <== osd.2 v2:172.21.15.123:6813/891966684 2 ==== command_reply(tid 2: 0 ) ==== 8+0+22 (crc 0 0 0) 0x7fa708009830 con 0x7fa6f4044c10 2024-09-22T04:06:48.675 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-22T04:06:48.675 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "" 2024-09-22T04:06:48.677 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-22T04:06:48.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.674+0000 7fa71867f640 1 -- 172.21.15.3:0/1614805270 >> v2:172.21.15.123:6813/891966684 conn(0x7fa6f4044c10 msgr2=0x7fa6f4047100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.674+0000 7fa71867f640 1 --2- 172.21.15.3:0/1614805270 >> v2:172.21.15.123:6813/891966684 conn(0x7fa6f4044c10 0x7fa6f4047100 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.674+0000 7fa71867f640 1 -- 172.21.15.3:0/1614805270 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa6f403ead0 msgr2=0x7fa6f4040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.674+0000 7fa71867f640 1 --2- 172.21.15.3:0/1614805270 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa6f403ead0 0x7fa6f4040fe0 crc :-1 s=READY pgs=3877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.674+0000 7fa71867f640 1 -- 172.21.15.3:0/1614805270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006eec0 msgr2=0x7fa710072990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.674+0000 7fa71867f640 1 --2- 172.21.15.3:0/1614805270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006eec0 0x7fa710072990 crc :-1 s=READY pgs=13028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.674+0000 7fa71867f640 1 -- 172.21.15.3:0/1614805270 shutdown_connections 2024-09-22T04:06:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.674+0000 7fa71867f640 1 --2- 172.21.15.3:0/1614805270 >> v2:172.21.15.123:6813/891966684 conn(0x7fa6f4044c10 0x7fa6f4047100 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.674+0000 7fa71867f640 1 --2- 172.21.15.3:0/1614805270 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fa6f403ead0 0x7fa6f4040fe0 unknown :-1 s=CLOSED pgs=3877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.674+0000 7fa71867f640 1 --2- 172.21.15.3:0/1614805270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa71006eec0 0x7fa710072990 unknown :-1 s=CLOSED pgs=13028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.674+0000 7fa71867f640 1 -- 172.21.15.3:0/1614805270 >> 172.21.15.3:0/1614805270 conn(0x7fa7100a39b0 msgr2=0x7fa7101baad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.674+0000 7fa71867f640 1 -- 172.21.15.3:0/1614805270 shutdown_connections 2024-09-22T04:06:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.674+0000 7fa71867f640 1 -- 172.21.15.3:0/1614805270 wait complete. 2024-09-22T04:06:48.685 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-22T04:06:48.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.793+0000 7f5983911640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:48.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.799+0000 7f5983911640 1 Processor -- start 2024-09-22T04:06:48.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.799+0000 7f5983911640 1 -- start start 2024-09-22T04:06:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.800+0000 7f5983911640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f597c0a3e10 0x7f597c0a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:48.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.801+0000 7f5983911640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f597c06ec50 con 0x7f597c0a4760 2024-09-22T04:06:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.801+0000 7f5983911640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f597c0a6130 con 0x7f597c0a3e10 2024-09-22T04:06:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.801+0000 7f59816ad640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f597c0a3e10 0x7f597c0a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.801+0000 7f5980eac640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f597c0a4760 0x7f597c135e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42790/0 (socket says 172.21.15.3:42790) 2024-09-22T04:06:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.801+0000 7f59816ad640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f597c0a3e10 0x7f597c0a4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34098/0 (socket says 172.21.15.3:34098) 2024-09-22T04:06:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.801+0000 7f5980eac640 1 -- 172.21.15.3:0/578245868 learned_addr learned my addr 172.21.15.3:0/578245868 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T04:06:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.801+0000 7f59816ad640 1 -- 172.21.15.3:0/578245868 >> v1:172.21.15.3:6789/0 conn(0x7f597c0a4760 legacy=0x7f597c135e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.801+0000 7f59816ad640 1 -- 172.21.15.3:0/578245868 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c0a5110 con 0x7f597c0a3e10 2024-09-22T04:06:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.801+0000 7f59816ad640 1 --2- 172.21.15.3:0/578245868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597c0a3e10 0x7f597c0a4220 crc :-1 s=READY pgs=13030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f21476a2754630 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.801+0000 7f5973fff640 1 -- 172.21.15.3:0/578245868 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5978009d90 con 0x7f597c0a3e10 2024-09-22T04:06:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.801+0000 7f5973fff640 1 -- 172.21.15.3:0/578245868 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f597800e230 con 0x7f597c0a3e10 2024-09-22T04:06:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.801+0000 7f5973fff640 1 -- 172.21.15.3:0/578245868 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f597800e4d0 con 0x7f597c0a3e10 2024-09-22T04:06:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.802+0000 7f5983911640 1 -- 172.21.15.3:0/578245868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597c0a3e10 msgr2=0x7f597c0a4220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.802+0000 7f5983911640 1 --2- 172.21.15.3:0/578245868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597c0a3e10 0x7f597c0a4220 crc :-1 s=READY pgs=13030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.802+0000 7f5983911640 1 -- 172.21.15.3:0/578245868 shutdown_connections 2024-09-22T04:06:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.802+0000 7f5983911640 1 --2- 172.21.15.3:0/578245868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597c0a3e10 0x7f597c0a4220 unknown :-1 s=CLOSED pgs=13030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.802+0000 7f5983911640 1 -- 172.21.15.3:0/578245868 >> 172.21.15.3:0/578245868 conn(0x7f597c0a74f0 msgr2=0x7f597c0a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.802+0000 7f5983911640 1 -- 172.21.15.3:0/578245868 shutdown_connections 2024-09-22T04:06:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.802+0000 7f5983911640 1 -- 172.21.15.3:0/578245868 wait complete. 2024-09-22T04:06:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.802+0000 7f5983911640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.804+0000 7f5983911640 1 Processor -- start 2024-09-22T04:06:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.804+0000 7f5983911640 1 -- start start 2024-09-22T04:06:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.806+0000 7f5983911640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597c0a3e10 0x7f597c123be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.806+0000 7f5983911640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f597c1278e0 con 0x7f597c0a3e10 2024-09-22T04:06:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.806+0000 7f59816ad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597c0a3e10 0x7f597c123be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.806+0000 7f59816ad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597c0a3e10 0x7f597c123be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34108/0 (socket says 172.21.15.3:34108) 2024-09-22T04:06:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.806+0000 7f59816ad640 1 -- 172.21.15.3:0/3907587328 learned_addr learned my addr 172.21.15.3:0/3907587328 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.806+0000 7f59816ad640 1 -- 172.21.15.3:0/3907587328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c125620 con 0x7f597c0a3e10 2024-09-22T04:06:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.806+0000 7f59816ad640 1 --2- 172.21.15.3:0/3907587328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597c0a3e10 0x7f597c123be0 crc :-1 s=READY pgs=13031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:48.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.806+0000 7f5971ffb640 1 -- 172.21.15.3:0/3907587328 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5978014750 con 0x7f597c0a3e10 2024-09-22T04:06:48.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.806+0000 7f5971ffb640 1 -- 172.21.15.3:0/3907587328 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f59780148b0 con 0x7f597c0a3e10 2024-09-22T04:06:48.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.806+0000 7f5983911640 1 -- 172.21.15.3:0/3907587328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597c124d70 con 0x7f597c0a3e10 2024-09-22T04:06:48.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.806+0000 7f5971ffb640 1 -- 172.21.15.3:0/3907587328 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f5978014b50 con 0x7f597c0a3e10 2024-09-22T04:06:48.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.807+0000 7f5983911640 1 -- 172.21.15.3:0/3907587328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f597c126f00 con 0x7f597c0a3e10 2024-09-22T04:06:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.808+0000 7f5983911640 1 -- 172.21.15.3:0/3907587328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f597c0a56d0 con 0x7f597c0a3e10 2024-09-22T04:06:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.808+0000 7f5971ffb640 1 -- 172.21.15.3:0/3907587328 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f5978014d00 con 0x7f597c0a3e10 2024-09-22T04:06:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.808+0000 7f5971ffb640 1 --2- 172.21.15.3:0/3907587328 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f596003eb20 0x7f5960041030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.808+0000 7f5971ffb640 1 -- 172.21.15.3:0/3907587328 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(733..733 src has 602..733) ==== 3431+0+0 (crc 0 0 0) 0x7f5978019040 con 0x7f597c0a3e10 2024-09-22T04:06:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.808+0000 7f5980eac640 1 --2- 172.21.15.3:0/3907587328 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f596003eb20 0x7f5960041030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:48.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.809+0000 7f5980eac640 1 --2- 172.21.15.3:0/3907587328 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f596003eb20 0x7f5960041030 crc :-1 s=READY pgs=3878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:48.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:48.813+0000 7f5971ffb640 1 -- 172.21.15.3:0/3907587328 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f5978011570 con 0x7f597c0a3e10 2024-09-22T04:06:49.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.005+0000 7f5983911640 1 -- 172.21.15.3:0/3907587328 --> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f597c0737f0 con 0x7f596003eb20 2024-09-22T04:06:49.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.006+0000 7f5971ffb640 1 -- 172.21.15.3:0/3907587328 <== mgr.4099 v2:172.21.15.123:6800/551784226 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (crc 0 0 0) 0x7f597c0737f0 con 0x7f596003eb20 2024-09-22T04:06:49.007 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 2 to deep-scrub 2024-09-22T04:06:49.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.008+0000 7f5983911640 1 -- 172.21.15.3:0/3907587328 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f596003eb20 msgr2=0x7f5960041030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:49.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.008+0000 7f5983911640 1 --2- 172.21.15.3:0/3907587328 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f596003eb20 0x7f5960041030 crc :-1 s=READY pgs=3878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.008+0000 7f5983911640 1 -- 172.21.15.3:0/3907587328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597c0a3e10 msgr2=0x7f597c123be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:49.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.008+0000 7f5983911640 1 --2- 172.21.15.3:0/3907587328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597c0a3e10 0x7f597c123be0 crc :-1 s=READY pgs=13031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.008+0000 7f5983911640 1 -- 172.21.15.3:0/3907587328 shutdown_connections 2024-09-22T04:06:49.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.008+0000 7f5983911640 1 --2- 172.21.15.3:0/3907587328 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f596003eb20 0x7f5960041030 unknown :-1 s=CLOSED pgs=3878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.008+0000 7f5983911640 1 --2- 172.21.15.3:0/3907587328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597c0a3e10 0x7f597c123be0 unknown :-1 s=CLOSED pgs=13031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.008+0000 7f5983911640 1 -- 172.21.15.3:0/3907587328 >> 172.21.15.3:0/3907587328 conn(0x7f597c0a74f0 msgr2=0x7f597c1bbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:49.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.008+0000 7f5983911640 1 -- 172.21.15.3:0/3907587328 shutdown_connections 2024-09-22T04:06:49.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.008+0000 7f5983911640 1 -- 172.21.15.3:0/3907587328 wait complete. 2024-09-22T04:06:49.021 INFO:tasks.ceph:Scrubbing osd.3 2024-09-22T04:06:49.021 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-22T04:06:49.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.132+0000 7efe2a391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:49.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.142+0000 7efe2a391640 1 Processor -- start 2024-09-22T04:06:49.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.142+0000 7efe2a391640 1 -- start start 2024-09-22T04:06:49.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.143+0000 7efe2a391640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efe2406eec0 0x7efe24138e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:49.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.143+0000 7efe2a391640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe240a4dc0 con 0x7efe2409dca0 2024-09-22T04:06:49.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.143+0000 7efe2a391640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7efe240a4ab0 con 0x7efe2406eec0 2024-09-22T04:06:49.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.143+0000 7efe237fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efe2406eec0 0x7efe24138e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:49.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.143+0000 7efe23fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7efe2409dca0 0x7efe2406e7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42800/0 (socket says 172.21.15.3:42800) 2024-09-22T04:06:49.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.143+0000 7efe23fff640 1 -- 172.21.15.3:0/661147273 learned_addr learned my addr 172.21.15.3:0/661147273 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T04:06:49.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.143+0000 7efe237fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efe2406eec0 0x7efe24138e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34122/0 (socket says 172.21.15.3:34122) 2024-09-22T04:06:49.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.144+0000 7efe237fe640 1 -- 172.21.15.3:0/661147273 >> v1:172.21.15.3:6789/0 conn(0x7efe2409dca0 legacy=0x7efe2406e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:49.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.144+0000 7efe22ffd640 1 -- 172.21.15.3:0/661147273 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 198261918 0 0) 0x7efe240a4dc0 con 0x7efe2409dca0 2024-09-22T04:06:49.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.144+0000 7efe237fe640 1 -- 172.21.15.3:0/661147273 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2409ef50 con 0x7efe2406eec0 2024-09-22T04:06:49.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.144+0000 7efe237fe640 1 --2- 172.21.15.3:0/661147273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe2406eec0 0x7efe24138e70 crc :-1 s=READY pgs=13033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b091a88b935cdd21 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:49.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.144+0000 7efe22ffd640 1 -- 172.21.15.3:0/661147273 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efe18009d90 con 0x7efe2406eec0 2024-09-22T04:06:49.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.144+0000 7efe22ffd640 1 -- 172.21.15.3:0/661147273 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7efe1800e230 con 0x7efe2406eec0 2024-09-22T04:06:49.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.144+0000 7efe22ffd640 1 -- 172.21.15.3:0/661147273 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efe1800e4d0 con 0x7efe2406eec0 2024-09-22T04:06:49.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.145+0000 7efe2a391640 1 -- 172.21.15.3:0/661147273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe2406eec0 msgr2=0x7efe24138e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:49.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.145+0000 7efe2a391640 1 --2- 172.21.15.3:0/661147273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe2406eec0 0x7efe24138e70 crc :-1 s=READY pgs=13033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.145+0000 7efe2a391640 1 -- 172.21.15.3:0/661147273 shutdown_connections 2024-09-22T04:06:49.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.145+0000 7efe2a391640 1 --2- 172.21.15.3:0/661147273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe2406eec0 0x7efe24138e70 unknown :-1 s=CLOSED pgs=13033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.145+0000 7efe2a391640 1 -- 172.21.15.3:0/661147273 >> 172.21.15.3:0/661147273 conn(0x7efe240a39b0 msgr2=0x7efe240a3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:49.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.145+0000 7efe2a391640 1 -- 172.21.15.3:0/661147273 shutdown_connections 2024-09-22T04:06:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.145+0000 7efe2a391640 1 -- 172.21.15.3:0/661147273 wait complete. 2024-09-22T04:06:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.145+0000 7efe2a391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.147+0000 7efe2a391640 1 Processor -- start 2024-09-22T04:06:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.147+0000 7efe2a391640 1 -- start start 2024-09-22T04:06:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.149+0000 7efe2a391640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe2406eec0 0x7efe24121610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.149+0000 7efe2a391640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efe24125290 con 0x7efe2406eec0 2024-09-22T04:06:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.149+0000 7efe23fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe2406eec0 0x7efe24121610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.149+0000 7efe23fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe2406eec0 0x7efe24121610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34128/0 (socket says 172.21.15.3:34128) 2024-09-22T04:06:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.149+0000 7efe23fff640 1 -- 172.21.15.3:0/3125675296 learned_addr learned my addr 172.21.15.3:0/3125675296 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.149+0000 7efe23fff640 1 -- 172.21.15.3:0/3125675296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe24123050 con 0x7efe2406eec0 2024-09-22T04:06:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.149+0000 7efe23fff640 1 --2- 172.21.15.3:0/3125675296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe2406eec0 0x7efe24121610 crc :-1 s=READY pgs=13034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:49.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.149+0000 7efe20ff9640 1 -- 172.21.15.3:0/3125675296 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efe14005a60 con 0x7efe2406eec0 2024-09-22T04:06:49.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.149+0000 7efe20ff9640 1 -- 172.21.15.3:0/3125675296 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7efe14005bc0 con 0x7efe2406eec0 2024-09-22T04:06:49.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.149+0000 7efe2a391640 1 -- 172.21.15.3:0/3125675296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe241227a0 con 0x7efe2406eec0 2024-09-22T04:06:49.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.149+0000 7efe20ff9640 1 -- 172.21.15.3:0/3125675296 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7efe14005e60 con 0x7efe2406eec0 2024-09-22T04:06:49.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.149+0000 7efe2a391640 1 -- 172.21.15.3:0/3125675296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe24123e70 con 0x7efe2406eec0 2024-09-22T04:06:49.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.150+0000 7efe2a391640 1 -- 172.21.15.3:0/3125675296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efe2409e3e0 con 0x7efe2406eec0 2024-09-22T04:06:49.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.151+0000 7efe20ff9640 1 -- 172.21.15.3:0/3125675296 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7efe140070b0 con 0x7efe2406eec0 2024-09-22T04:06:49.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.151+0000 7efe20ff9640 1 --2- 172.21.15.3:0/3125675296 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7efe1003ead0 0x7efe10040fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:49.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.151+0000 7efe20ff9640 1 -- 172.21.15.3:0/3125675296 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(733..733 src has 602..733) ==== 3431+0+0 (crc 0 0 0) 0x7efe1404fac0 con 0x7efe2406eec0 2024-09-22T04:06:49.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.151+0000 7efe20ff9640 1 --2- 172.21.15.3:0/3125675296 >> v2:172.21.15.123:6806/4206675901 conn(0x7efe10044c10 0x7efe10047100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:49.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.151+0000 7efe20ff9640 1 -- 172.21.15.3:0/3125675296 --> v2:172.21.15.123:6806/4206675901 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efe10047790 con 0x7efe10044c10 2024-09-22T04:06:49.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.151+0000 7efe20ff9640 1 -- 172.21.15.3:0/3125675296 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=733) ==== 24+0+0 (crc 0 0 0) 0x7efe1404fed0 con 0x7efe2406eec0 2024-09-22T04:06:49.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.151+0000 7efe237fe640 1 --2- 172.21.15.3:0/3125675296 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7efe1003ead0 0x7efe10040fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:49.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.152+0000 7efe237fe640 1 --2- 172.21.15.3:0/3125675296 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7efe1003ead0 0x7efe10040fe0 crc :-1 s=READY pgs=3879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:49.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.152+0000 7efe2892e640 1 --2- 172.21.15.3:0/3125675296 >> v2:172.21.15.123:6806/4206675901 conn(0x7efe10044c10 0x7efe10047100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-09-22T04:06:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.155+0000 7efe2892e640 1 --2- 172.21.15.3:0/3125675296 >> v2:172.21.15.123:6806/4206675901 conn(0x7efe10044c10 0x7efe10047100 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:49.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.156+0000 7efe20ff9640 1 -- 172.21.15.3:0/3125675296 <== osd.3 v2:172.21.15.123:6806/4206675901 1 ==== command_reply(tid 1: 0 ) ==== 8+0+5790 (crc 0 0 0) 0x7efe10047790 con 0x7efe10044c10 2024-09-22T04:06:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.159+0000 7efe2a391640 1 -- 172.21.15.3:0/3125675296 --> v2:172.21.15.123:6806/4206675901 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efe241254a0 con 0x7efe10044c10 2024-09-22T04:06:49.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.161+0000 7efe20ff9640 1 -- 172.21.15.3:0/3125675296 <== osd.3 v2:172.21.15.123:6806/4206675901 2 ==== command_reply(tid 2: 0 ) ==== 8+0+22 (crc 0 0 0) 0x7efe1c00a820 con 0x7efe10044c10 2024-09-22T04:06:49.162 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-22T04:06:49.162 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "" 2024-09-22T04:06:49.162 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-22T04:06:49.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.161+0000 7efe2a391640 1 -- 172.21.15.3:0/3125675296 >> v2:172.21.15.123:6806/4206675901 conn(0x7efe10044c10 msgr2=0x7efe10047100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:49.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.161+0000 7efe2a391640 1 --2- 172.21.15.3:0/3125675296 >> v2:172.21.15.123:6806/4206675901 conn(0x7efe10044c10 0x7efe10047100 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.161+0000 7efe2a391640 1 -- 172.21.15.3:0/3125675296 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7efe1003ead0 msgr2=0x7efe10040fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:49.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.161+0000 7efe2a391640 1 --2- 172.21.15.3:0/3125675296 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7efe1003ead0 0x7efe10040fe0 crc :-1 s=READY pgs=3879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.161+0000 7efe2a391640 1 -- 172.21.15.3:0/3125675296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe2406eec0 msgr2=0x7efe24121610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:49.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.161+0000 7efe2a391640 1 --2- 172.21.15.3:0/3125675296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe2406eec0 0x7efe24121610 crc :-1 s=READY pgs=13034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.161+0000 7efe2a391640 1 -- 172.21.15.3:0/3125675296 shutdown_connections 2024-09-22T04:06:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.161+0000 7efe2a391640 1 --2- 172.21.15.3:0/3125675296 >> v2:172.21.15.123:6806/4206675901 conn(0x7efe10044c10 0x7efe10047100 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.161+0000 7efe2a391640 1 --2- 172.21.15.3:0/3125675296 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7efe1003ead0 0x7efe10040fe0 unknown :-1 s=CLOSED pgs=3879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.161+0000 7efe2a391640 1 --2- 172.21.15.3:0/3125675296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe2406eec0 0x7efe24121610 unknown :-1 s=CLOSED pgs=13034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.161+0000 7efe2a391640 1 -- 172.21.15.3:0/3125675296 >> 172.21.15.3:0/3125675296 conn(0x7efe240a39b0 msgr2=0x7efe241bab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.161+0000 7efe2a391640 1 -- 172.21.15.3:0/3125675296 shutdown_connections 2024-09-22T04:06:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.161+0000 7efe2a391640 1 -- 172.21.15.3:0/3125675296 wait complete. 2024-09-22T04:06:49.173 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-22T04:06:49.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.280+0000 7fc554751640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:49.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.287+0000 7fc554751640 1 Processor -- start 2024-09-22T04:06:49.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.287+0000 7fc554751640 1 -- start start 2024-09-22T04:06:49.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.288+0000 7fc554751640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc54c0a3e10 0x7fc54c0a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:49.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.288+0000 7fc554751640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc54c06ec50 con 0x7fc54c0a4760 2024-09-22T04:06:49.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.288+0000 7fc554751640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc54c0a6130 con 0x7fc54c0a3e10 2024-09-22T04:06:49.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.289+0000 7fc5524ed640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc54c0a3e10 0x7fc54c0a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:49.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.289+0000 7fc551cec640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc54c0a4760 0x7fc54c135e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42808/0 (socket says 172.21.15.3:42808) 2024-09-22T04:06:49.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.289+0000 7fc5524ed640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc54c0a3e10 0x7fc54c0a4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34140/0 (socket says 172.21.15.3:34140) 2024-09-22T04:06:49.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.289+0000 7fc551cec640 1 -- 172.21.15.3:0/2101908515 learned_addr learned my addr 172.21.15.3:0/2101908515 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T04:06:49.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.289+0000 7fc5524ed640 1 -- 172.21.15.3:0/2101908515 >> v1:172.21.15.3:6789/0 conn(0x7fc54c0a4760 legacy=0x7fc54c135e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:49.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.289+0000 7fc5524ed640 1 -- 172.21.15.3:0/2101908515 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c0a5110 con 0x7fc54c0a3e10 2024-09-22T04:06:49.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.289+0000 7fc5514eb640 1 -- 172.21.15.3:0/2101908515 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2389803680 0 0) 0x7fc54c06ec50 con 0x7fc54c0a4760 2024-09-22T04:06:49.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.289+0000 7fc5524ed640 1 --2- 172.21.15.3:0/2101908515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54c0a3e10 0x7fc54c0a4220 crc :-1 s=READY pgs=13036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5058222a55906f server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:49.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.289+0000 7fc5514eb640 1 -- 172.21.15.3:0/2101908515 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc54800e040 con 0x7fc54c0a3e10 2024-09-22T04:06:49.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.289+0000 7fc5514eb640 1 -- 172.21.15.3:0/2101908515 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc548007670 con 0x7fc54c0a3e10 2024-09-22T04:06:49.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.289+0000 7fc5514eb640 1 -- 172.21.15.3:0/2101908515 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc548007910 con 0x7fc54c0a3e10 2024-09-22T04:06:49.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.290+0000 7fc554751640 1 -- 172.21.15.3:0/2101908515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54c0a3e10 msgr2=0x7fc54c0a4220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:49.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.290+0000 7fc554751640 1 --2- 172.21.15.3:0/2101908515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54c0a3e10 0x7fc54c0a4220 crc :-1 s=READY pgs=13036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.290+0000 7fc554751640 1 -- 172.21.15.3:0/2101908515 shutdown_connections 2024-09-22T04:06:49.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.290+0000 7fc554751640 1 --2- 172.21.15.3:0/2101908515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54c0a3e10 0x7fc54c0a4220 unknown :-1 s=CLOSED pgs=13036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.290+0000 7fc554751640 1 -- 172.21.15.3:0/2101908515 >> 172.21.15.3:0/2101908515 conn(0x7fc54c0a74f0 msgr2=0x7fc54c0a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:49.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.290+0000 7fc554751640 1 -- 172.21.15.3:0/2101908515 shutdown_connections 2024-09-22T04:06:49.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.290+0000 7fc554751640 1 -- 172.21.15.3:0/2101908515 wait complete. 2024-09-22T04:06:49.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.290+0000 7fc554751640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:49.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.292+0000 7fc554751640 1 Processor -- start 2024-09-22T04:06:49.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.292+0000 7fc554751640 1 -- start start 2024-09-22T04:06:49.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.294+0000 7fc554751640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54c0a3e10 0x7fc54c123df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:49.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.294+0000 7fc554751640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c126070 con 0x7fc54c0a3e10 2024-09-22T04:06:49.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.294+0000 7fc5524ed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54c0a3e10 0x7fc54c123df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:49.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.294+0000 7fc5524ed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54c0a3e10 0x7fc54c123df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34146/0 (socket says 172.21.15.3:34146) 2024-09-22T04:06:49.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.294+0000 7fc5524ed640 1 -- 172.21.15.3:0/3031863754 learned_addr learned my addr 172.21.15.3:0/3031863754 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:49.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.294+0000 7fc5524ed640 1 -- 172.21.15.3:0/3031863754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c1251c0 con 0x7fc54c0a3e10 2024-09-22T04:06:49.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.294+0000 7fc5524ed640 1 --2- 172.21.15.3:0/3031863754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54c0a3e10 0x7fc54c123df0 crc :-1 s=READY pgs=13037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:49.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.294+0000 7fc542ffd640 1 -- 172.21.15.3:0/3031863754 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc54800e040 con 0x7fc54c0a3e10 2024-09-22T04:06:49.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.294+0000 7fc542ffd640 1 -- 172.21.15.3:0/3031863754 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fc548013aa0 con 0x7fc54c0a3e10 2024-09-22T04:06:49.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.294+0000 7fc554751640 1 -- 172.21.15.3:0/3031863754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54c123780 con 0x7fc54c0a3e10 2024-09-22T04:06:49.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.294+0000 7fc542ffd640 1 -- 172.21.15.3:0/3031863754 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fc548013d40 con 0x7fc54c0a3e10 2024-09-22T04:06:49.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.294+0000 7fc554751640 1 -- 172.21.15.3:0/3031863754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54c123a50 con 0x7fc54c0a3e10 2024-09-22T04:06:49.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.295+0000 7fc554751640 1 -- 172.21.15.3:0/3031863754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54c125ae0 con 0x7fc54c0a3e10 2024-09-22T04:06:49.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.295+0000 7fc542ffd640 1 -- 172.21.15.3:0/3031863754 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fc548011040 con 0x7fc54c0a3e10 2024-09-22T04:06:49.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.296+0000 7fc542ffd640 1 --2- 172.21.15.3:0/3031863754 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc53003eb00 0x7fc530041010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:49.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.296+0000 7fc542ffd640 1 -- 172.21.15.3:0/3031863754 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(733..733 src has 602..733) ==== 3431+0+0 (crc 0 0 0) 0x7fc5480515b0 con 0x7fc54c0a3e10 2024-09-22T04:06:49.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.296+0000 7fc551cec640 1 --2- 172.21.15.3:0/3031863754 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc53003eb00 0x7fc530041010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:49.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.297+0000 7fc551cec640 1 --2- 172.21.15.3:0/3031863754 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc53003eb00 0x7fc530041010 crc :-1 s=READY pgs=3880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:49.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.301+0000 7fc542ffd640 1 -- 172.21.15.3:0/3031863754 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7fc548015580 con 0x7fc54c0a3e10 2024-09-22T04:06:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.491+0000 7fc554751640 1 -- 172.21.15.3:0/3031863754 --> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fc54c127850 con 0x7fc53003eb00 2024-09-22T04:06:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.491+0000 7fc542ffd640 1 -- 172.21.15.3:0/3031863754 <== mgr.4099 v2:172.21.15.123:6800/551784226 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (crc 0 0 0) 0x7fc54c127850 con 0x7fc53003eb00 2024-09-22T04:06:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 3 to deep-scrub 2024-09-22T04:06:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.493+0000 7fc554751640 1 -- 172.21.15.3:0/3031863754 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc53003eb00 msgr2=0x7fc530041010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.493+0000 7fc554751640 1 --2- 172.21.15.3:0/3031863754 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc53003eb00 0x7fc530041010 crc :-1 s=READY pgs=3880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.493+0000 7fc554751640 1 -- 172.21.15.3:0/3031863754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54c0a3e10 msgr2=0x7fc54c123df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.493+0000 7fc554751640 1 --2- 172.21.15.3:0/3031863754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54c0a3e10 0x7fc54c123df0 crc :-1 s=READY pgs=13037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.493+0000 7fc554751640 1 -- 172.21.15.3:0/3031863754 shutdown_connections 2024-09-22T04:06:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.493+0000 7fc554751640 1 --2- 172.21.15.3:0/3031863754 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fc53003eb00 0x7fc530041010 unknown :-1 s=CLOSED pgs=3880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.493+0000 7fc554751640 1 --2- 172.21.15.3:0/3031863754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc54c0a3e10 0x7fc54c123df0 unknown :-1 s=CLOSED pgs=13037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.493+0000 7fc554751640 1 -- 172.21.15.3:0/3031863754 >> 172.21.15.3:0/3031863754 conn(0x7fc54c0a74f0 msgr2=0x7fc54c0713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.494+0000 7fc554751640 1 -- 172.21.15.3:0/3031863754 shutdown_connections 2024-09-22T04:06:49.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.494+0000 7fc554751640 1 -- 172.21.15.3:0/3031863754 wait complete. 2024-09-22T04:06:49.506 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T04:06:49.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.618+0000 7f1842d51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:49.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.623+0000 7f1842d51640 1 Processor -- start 2024-09-22T04:06:49.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.623+0000 7f1842d51640 1 -- start start 2024-09-22T04:06:49.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.625+0000 7f1842d51640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f183c138ef0 0x7f183c13b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:49.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.625+0000 7f1842d51640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f183c09efe0 con 0x7f183c136070 2024-09-22T04:06:49.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.625+0000 7f1842d51640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f183c135c20 con 0x7f183c138ef0 2024-09-22T04:06:49.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.625+0000 7f183bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f183c138ef0 0x7f183c13b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:49.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.625+0000 7f1840aed640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f183c136070 0x7f183c136480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:42814/0 (socket says 172.21.15.3:42814) 2024-09-22T04:06:49.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.625+0000 7f183bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f183c138ef0 0x7f183c13b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34156/0 (socket says 172.21.15.3:34156) 2024-09-22T04:06:49.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.625+0000 7f1840aed640 1 -- 172.21.15.3:0/2231682982 learned_addr learned my addr 172.21.15.3:0/2231682982 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T04:06:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.625+0000 7f183bfff640 1 -- 172.21.15.3:0/2231682982 >> v1:172.21.15.3:6789/0 conn(0x7f183c136070 legacy=0x7f183c136480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.625+0000 7f183bfff640 1 -- 172.21.15.3:0/2231682982 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c09d360 con 0x7f183c138ef0 2024-09-22T04:06:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.626+0000 7f183bfff640 1 --2- 172.21.15.3:0/2231682982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c138ef0 0x7f183c13b310 crc :-1 s=READY pgs=13039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e3495c203efc6d server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.626+0000 7f183b7fe640 1 -- 172.21.15.3:0/2231682982 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f1830009d80 con 0x7f183c138ef0 2024-09-22T04:06:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.626+0000 7f183b7fe640 1 -- 172.21.15.3:0/2231682982 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f183000e230 con 0x7f183c138ef0 2024-09-22T04:06:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.626+0000 7f183b7fe640 1 -- 172.21.15.3:0/2231682982 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f183000e4d0 con 0x7f183c138ef0 2024-09-22T04:06:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.626+0000 7f1842d51640 1 -- 172.21.15.3:0/2231682982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c138ef0 msgr2=0x7f183c13b310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:49.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.626+0000 7f1842d51640 1 --2- 172.21.15.3:0/2231682982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c138ef0 0x7f183c13b310 crc :-1 s=READY pgs=13039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.627+0000 7f1842d51640 1 -- 172.21.15.3:0/2231682982 shutdown_connections 2024-09-22T04:06:49.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.627+0000 7f1842d51640 1 --2- 172.21.15.3:0/2231682982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c138ef0 0x7f183c13b310 unknown :-1 s=CLOSED pgs=13039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.627+0000 7f1842d51640 1 -- 172.21.15.3:0/2231682982 >> 172.21.15.3:0/2231682982 conn(0x7f183c06eb10 msgr2=0x7f183c06ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:49.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.627+0000 7f1842d51640 1 -- 172.21.15.3:0/2231682982 shutdown_connections 2024-09-22T04:06:49.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.627+0000 7f1842d51640 1 -- 172.21.15.3:0/2231682982 wait complete. 2024-09-22T04:06:49.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.627+0000 7f1842d51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:06:49.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.629+0000 7f1842d51640 1 Processor -- start 2024-09-22T04:06:49.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.629+0000 7f1842d51640 1 -- start start 2024-09-22T04:06:49.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.630+0000 7f1842d51640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c136070 0x7f183c133190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:49.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.630+0000 7f1842d51640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f183c123170 con 0x7f183c136070 2024-09-22T04:06:49.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.630+0000 7f1840aed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c136070 0x7f183c133190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:49.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.630+0000 7f1840aed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c136070 0x7f183c133190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34170/0 (socket says 172.21.15.3:34170) 2024-09-22T04:06:49.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.630+0000 7f1840aed640 1 -- 172.21.15.3:0/2400136405 learned_addr learned my addr 172.21.15.3:0/2400136405 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:06:49.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.631+0000 7f1840aed640 1 -- 172.21.15.3:0/2400136405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c122e40 con 0x7f183c136070 2024-09-22T04:06:49.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.631+0000 7f1840aed640 1 --2- 172.21.15.3:0/2400136405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c136070 0x7f183c133190 crc :-1 s=READY pgs=13040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:49.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.631+0000 7f18397fa640 1 -- 172.21.15.3:0/2400136405 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f182c005a60 con 0x7f183c136070 2024-09-22T04:06:49.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.631+0000 7f18397fa640 1 -- 172.21.15.3:0/2400136405 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7f182c005bc0 con 0x7f183c136070 2024-09-22T04:06:49.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.631+0000 7f1842d51640 1 -- 172.21.15.3:0/2400136405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183c0a3c70 con 0x7f183c136070 2024-09-22T04:06:49.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.631+0000 7f18397fa640 1 -- 172.21.15.3:0/2400136405 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7f182c005e60 con 0x7f183c136070 2024-09-22T04:06:49.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.631+0000 7f1842d51640 1 -- 172.21.15.3:0/2400136405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183c122590 con 0x7f183c136070 2024-09-22T04:06:49.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.632+0000 7f1842d51640 1 -- 172.21.15.3:0/2400136405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183c06a460 con 0x7f183c136070 2024-09-22T04:06:49.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.632+0000 7f18397fa640 1 -- 172.21.15.3:0/2400136405 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7f182c0070b0 con 0x7f183c136070 2024-09-22T04:06:49.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.632+0000 7f18397fa640 1 --2- 172.21.15.3:0/2400136405 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f182803ea80 0x7f1828040f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:06:49.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.632+0000 7f18397fa640 1 -- 172.21.15.3:0/2400136405 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(733..733 src has 602..733) ==== 3431+0+0 (crc 0 0 0) 0x7f182c04fba0 con 0x7f183c136070 2024-09-22T04:06:49.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.633+0000 7f183bfff640 1 --2- 172.21.15.3:0/2400136405 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f182803ea80 0x7f1828040f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:06:49.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.633+0000 7f183bfff640 1 --2- 172.21.15.3:0/2400136405 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f182803ea80 0x7f1828040f90 crc :-1 s=READY pgs=3881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:06:49.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.638+0000 7f18397fa640 1 -- 172.21.15.3:0/2400136405 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7f182c00dc10 con 0x7f183c136070 2024-09-22T04:06:49.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.823+0000 7f1842d51640 1 -- 172.21.15.3:0/2400136405 --> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f183c09d990 con 0x7f182803ea80 2024-09-22T04:06:49.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.825+0000 7f18397fa640 1 -- 172.21.15.3:0/2400136405 <== mgr.4099 v2:172.21.15.123:6800/551784226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+109430 (crc 0 0 0) 0x7f1830009e70 con 0x7f182803ea80 2024-09-22T04:06:49.827 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T04:06:49.828 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-22T04:06:49.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.828+0000 7f1842d51640 1 -- 172.21.15.3:0/2400136405 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f182803ea80 msgr2=0x7f1828040f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:49.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.828+0000 7f1842d51640 1 --2- 172.21.15.3:0/2400136405 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f182803ea80 0x7f1828040f90 crc :-1 s=READY pgs=3881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.828+0000 7f1842d51640 1 -- 172.21.15.3:0/2400136405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c136070 msgr2=0x7f183c133190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:06:49.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.828+0000 7f1842d51640 1 --2- 172.21.15.3:0/2400136405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c136070 0x7f183c133190 crc :-1 s=READY pgs=13040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.828+0000 7f1842d51640 1 -- 172.21.15.3:0/2400136405 shutdown_connections 2024-09-22T04:06:49.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.828+0000 7f1842d51640 1 --2- 172.21.15.3:0/2400136405 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7f182803ea80 0x7f1828040f90 unknown :-1 s=CLOSED pgs=3881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.828+0000 7f1842d51640 1 --2- 172.21.15.3:0/2400136405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c136070 0x7f183c133190 unknown :-1 s=CLOSED pgs=13040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:06:49.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.828+0000 7f1842d51640 1 -- 172.21.15.3:0/2400136405 >> 172.21.15.3:0/2400136405 conn(0x7f183c06eb10 msgr2=0x7f183c1be010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:06:49.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.828+0000 7f1842d51640 1 -- 172.21.15.3:0/2400136405 shutdown_connections 2024-09-22T04:06:49.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:06:49.828+0000 7f1842d51640 1 -- 172.21.15.3:0/2400136405 wait complete. 2024-09-22T04:06:49.840 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":1326,"stamp":"2024-09-22T04:06:47.842102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590392,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":52,"num_write":88,"num_write_kb":1115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62,"ondisk_log_size":62,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":17344,"kb_used_data":17344,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209173568,"statfs":{"total":214211493888,"available":214193733632,"internally_reserved":0,"allocated":17760256,"data_stored":17760256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-321875,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-25,"num_write_kb":-325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001843"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.468761+0000","last_change":"2024-09-22T03:52:13.461984+0000","last_active":"2024-09-22T04:06:44.468761+0000","last_peered":"2024-09-22T04:06:44.468761+0000","last_clean":"2024-09-22T04:06:44.468761+0000","last_became_active":"2024-09-22T03:52:13.436104+0000","last_became_peered":"2024-09-22T03:52:13.436104+0000","last_unstale":"2024-09-22T04:06:44.468761+0000","last_undegraded":"2024-09-22T04:06:44.468761+0000","last_fullsized":"2024-09-22T04:06:44.468761+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.531561+0000","last_change":"2024-09-22T03:52:13.517533+0000","last_active":"2024-09-22T04:06:44.531561+0000","last_peered":"2024-09-22T04:06:44.531561+0000","last_clean":"2024-09-22T04:06:44.531561+0000","last_became_active":"2024-09-22T03:52:13.485941+0000","last_became_peered":"2024-09-22T03:52:13.485941+0000","last_unstale":"2024-09-22T04:06:44.531561+0000","last_undegraded":"2024-09-22T04:06:44.531561+0000","last_fullsized":"2024-09-22T04:06:44.531561+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.476678+0000","last_change":"2024-09-22T03:52:13.441983+0000","last_active":"2024-09-22T04:06:44.476678+0000","last_peered":"2024-09-22T04:06:44.476678+0000","last_clean":"2024-09-22T04:06:44.476678+0000","last_became_active":"2024-09-22T03:52:13.435054+0000","last_became_peered":"2024-09-22T03:52:13.435054+0000","last_unstale":"2024-09-22T04:06:44.476678+0000","last_undegraded":"2024-09-22T04:06:44.476678+0000","last_fullsized":"2024-09-22T04:06:44.476678+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.484441+0000","last_change":"2024-09-22T03:52:13.475862+0000","last_active":"2024-09-22T04:06:44.484441+0000","last_peered":"2024-09-22T04:06:44.484441+0000","last_clean":"2024-09-22T04:06:44.484441+0000","last_became_active":"2024-09-22T03:52:13.424125+0000","last_became_peered":"2024-09-22T03:52:13.424125+0000","last_unstale":"2024-09-22T04:06:44.484441+0000","last_undegraded":"2024-09-22T04:06:44.484441+0000","last_fullsized":"2024-09-22T04:06:44.484441+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.489151+0000","last_change":"2024-09-22T03:52:13.638357+0000","last_active":"2024-09-22T04:06:44.489151+0000","last_peered":"2024-09-22T04:06:44.489151+0000","last_clean":"2024-09-22T04:06:44.489151+0000","last_became_active":"2024-09-22T03:52:13.612276+0000","last_became_peered":"2024-09-22T03:52:13.612276+0000","last_unstale":"2024-09-22T04:06:44.489151+0000","last_undegraded":"2024-09-22T04:06:44.489151+0000","last_fullsized":"2024-09-22T04:06:44.489151+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.482802+0000","last_change":"2024-09-22T03:52:13.631942+0000","last_active":"2024-09-22T04:06:44.482802+0000","last_peered":"2024-09-22T04:06:44.482802+0000","last_clean":"2024-09-22T04:06:44.482802+0000","last_became_active":"2024-09-22T03:52:13.584449+0000","last_became_peered":"2024-09-22T03:52:13.584449+0000","last_unstale":"2024-09-22T04:06:44.482802+0000","last_undegraded":"2024-09-22T04:06:44.482802+0000","last_fullsized":"2024-09-22T04:06:44.482802+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.492811+0000","last_change":"2024-09-22T03:52:13.462393+0000","last_active":"2024-09-22T04:06:44.492811+0000","last_peered":"2024-09-22T04:06:44.492811+0000","last_clean":"2024-09-22T04:06:44.492811+0000","last_became_active":"2024-09-22T03:52:13.438920+0000","last_became_peered":"2024-09-22T03:52:13.438920+0000","last_unstale":"2024-09-22T04:06:44.492811+0000","last_undegraded":"2024-09-22T04:06:44.492811+0000","last_fullsized":"2024-09-22T04:06:44.492811+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.481246+0000","last_change":"2024-09-22T03:52:13.435067+0000","last_active":"2024-09-22T04:06:44.481246+0000","last_peered":"2024-09-22T04:06:44.481246+0000","last_clean":"2024-09-22T04:06:44.481246+0000","last_became_active":"2024-09-22T03:52:13.424399+0000","last_became_peered":"2024-09-22T03:52:13.424399+0000","last_unstale":"2024-09-22T04:06:44.481246+0000","last_undegraded":"2024-09-22T04:06:44.481246+0000","last_fullsized":"2024-09-22T04:06:44.481246+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.456201+0000","last_change":"2024-09-22T03:52:13.414909+0000","last_active":"2024-09-22T04:06:44.456201+0000","last_peered":"2024-09-22T04:06:44.456201+0000","last_clean":"2024-09-22T04:06:44.456201+0000","last_became_active":"2024-09-22T03:52:13.367950+0000","last_became_peered":"2024-09-22T03:52:13.367950+0000","last_unstale":"2024-09-22T04:06:44.456201+0000","last_undegraded":"2024-09-22T04:06:44.456201+0000","last_fullsized":"2024-09-22T04:06:44.456201+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.463555+0000","last_change":"2024-09-22T03:52:13.421431+0000","last_active":"2024-09-22T04:06:44.463555+0000","last_peered":"2024-09-22T04:06:44.463555+0000","last_clean":"2024-09-22T04:06:44.463555+0000","last_became_active":"2024-09-22T03:52:13.370886+0000","last_became_peered":"2024-09-22T03:52:13.370886+0000","last_unstale":"2024-09-22T04:06:44.463555+0000","last_undegraded":"2024-09-22T04:06:44.463555+0000","last_fullsized":"2024-09-22T04:06:44.463555+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.469081+0000","last_change":"2024-09-22T03:52:13.488521+0000","last_active":"2024-09-22T04:06:44.469081+0000","last_peered":"2024-09-22T04:06:44.469081+0000","last_clean":"2024-09-22T04:06:44.469081+0000","last_became_active":"2024-09-22T03:52:13.477618+0000","last_became_peered":"2024-09-22T03:52:13.477618+0000","last_unstale":"2024-09-22T04:06:44.469081+0000","last_undegraded":"2024-09-22T04:06:44.469081+0000","last_fullsized":"2024-09-22T04:06:44.469081+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.477943+0000","last_change":"2024-09-22T03:52:13.423108+0000","last_active":"2024-09-22T04:06:44.477943+0000","last_peered":"2024-09-22T04:06:44.477943+0000","last_clean":"2024-09-22T04:06:44.477943+0000","last_became_active":"2024-09-22T03:52:13.354826+0000","last_became_peered":"2024-09-22T03:52:13.354826+0000","last_unstale":"2024-09-22T04:06:44.477943+0000","last_undegraded":"2024-09-22T04:06:44.477943+0000","last_fullsized":"2024-09-22T04:06:44.477943+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.461834+0000","last_change":"2024-09-22T03:52:13.456498+0000","last_active":"2024-09-22T04:06:44.461834+0000","last_peered":"2024-09-22T04:06:44.461834+0000","last_clean":"2024-09-22T04:06:44.461834+0000","last_became_active":"2024-09-22T03:52:13.405184+0000","last_became_peered":"2024-09-22T03:52:13.405184+0000","last_unstale":"2024-09-22T04:06:44.461834+0000","last_undegraded":"2024-09-22T04:06:44.461834+0000","last_fullsized":"2024-09-22T04:06:44.461834+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.477816+0000","last_change":"2024-09-22T03:52:13.548435+0000","last_active":"2024-09-22T04:06:44.477816+0000","last_peered":"2024-09-22T04:06:44.477816+0000","last_clean":"2024-09-22T04:06:44.477816+0000","last_became_active":"2024-09-22T03:52:13.533286+0000","last_became_peered":"2024-09-22T03:52:13.533286+0000","last_unstale":"2024-09-22T04:06:44.477816+0000","last_undegraded":"2024-09-22T04:06:44.477816+0000","last_fullsized":"2024-09-22T04:06:44.477816+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.476662+0000","last_change":"2024-09-22T03:52:13.555314+0000","last_active":"2024-09-22T04:06:44.476662+0000","last_peered":"2024-09-22T04:06:44.476662+0000","last_clean":"2024-09-22T04:06:44.476662+0000","last_became_active":"2024-09-22T03:52:13.531887+0000","last_became_peered":"2024-09-22T03:52:13.531887+0000","last_unstale":"2024-09-22T04:06:44.476662+0000","last_undegraded":"2024-09-22T04:06:44.476662+0000","last_fullsized":"2024-09-22T04:06:44.476662+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.538057+0000","last_change":"2024-09-22T03:52:13.576998+0000","last_active":"2024-09-22T04:06:44.538057+0000","last_peered":"2024-09-22T04:06:44.538057+0000","last_clean":"2024-09-22T04:06:44.538057+0000","last_became_active":"2024-09-22T03:52:13.562231+0000","last_became_peered":"2024-09-22T03:52:13.562231+0000","last_unstale":"2024-09-22T04:06:44.538057+0000","last_undegraded":"2024-09-22T04:06:44.538057+0000","last_fullsized":"2024-09-22T04:06:44.538057+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.480181+0000","last_change":"2024-09-22T03:52:13.527047+0000","last_active":"2024-09-22T04:06:44.480181+0000","last_peered":"2024-09-22T04:06:44.480181+0000","last_clean":"2024-09-22T04:06:44.480181+0000","last_became_active":"2024-09-22T03:52:13.496766+0000","last_became_peered":"2024-09-22T03:52:13.496766+0000","last_unstale":"2024-09-22T04:06:44.480181+0000","last_undegraded":"2024-09-22T04:06:44.480181+0000","last_fullsized":"2024-09-22T04:06:44.480181+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.482475+0000","last_change":"2024-09-22T03:52:13.517430+0000","last_active":"2024-09-22T04:06:44.482475+0000","last_peered":"2024-09-22T04:06:44.482475+0000","last_clean":"2024-09-22T04:06:44.482475+0000","last_became_active":"2024-09-22T03:52:13.496886+0000","last_became_peered":"2024-09-22T03:52:13.496886+0000","last_unstale":"2024-09-22T04:06:44.482475+0000","last_undegraded":"2024-09-22T04:06:44.482475+0000","last_fullsized":"2024-09-22T04:06:44.482475+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.490055+0000","last_change":"2024-09-22T03:52:13.464870+0000","last_active":"2024-09-22T04:06:44.490055+0000","last_peered":"2024-09-22T04:06:44.490055+0000","last_clean":"2024-09-22T04:06:44.490055+0000","last_became_active":"2024-09-22T03:52:13.412673+0000","last_became_peered":"2024-09-22T03:52:13.412673+0000","last_unstale":"2024-09-22T04:06:44.490055+0000","last_undegraded":"2024-09-22T04:06:44.490055+0000","last_fullsized":"2024-09-22T04:06:44.490055+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.478723+0000","last_change":"2024-09-22T03:52:13.477357+0000","last_active":"2024-09-22T04:06:44.478723+0000","last_peered":"2024-09-22T04:06:44.478723+0000","last_clean":"2024-09-22T04:06:44.478723+0000","last_became_active":"2024-09-22T03:52:13.441995+0000","last_became_peered":"2024-09-22T03:52:13.441995+0000","last_unstale":"2024-09-22T04:06:44.478723+0000","last_undegraded":"2024-09-22T04:06:44.478723+0000","last_fullsized":"2024-09-22T04:06:44.478723+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.463150+0000","last_change":"2024-09-22T03:52:13.667385+0000","last_active":"2024-09-22T04:06:44.463150+0000","last_peered":"2024-09-22T04:06:44.463150+0000","last_clean":"2024-09-22T04:06:44.463150+0000","last_became_active":"2024-09-22T03:52:13.657594+0000","last_became_peered":"2024-09-22T03:52:13.657594+0000","last_unstale":"2024-09-22T04:06:44.463150+0000","last_undegraded":"2024-09-22T04:06:44.463150+0000","last_fullsized":"2024-09-22T04:06:44.463150+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.474456+0000","last_change":"2024-09-22T03:52:13.488158+0000","last_active":"2024-09-22T04:06:44.474456+0000","last_peered":"2024-09-22T04:06:44.474456+0000","last_clean":"2024-09-22T04:06:44.474456+0000","last_became_active":"2024-09-22T03:52:13.435529+0000","last_became_peered":"2024-09-22T03:52:13.435529+0000","last_unstale":"2024-09-22T04:06:44.474456+0000","last_undegraded":"2024-09-22T04:06:44.474456+0000","last_fullsized":"2024-09-22T04:06:44.474456+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.477411+0000","last_change":"2024-09-22T03:52:13.418644+0000","last_active":"2024-09-22T04:06:44.477411+0000","last_peered":"2024-09-22T04:06:44.477411+0000","last_clean":"2024-09-22T04:06:44.477411+0000","last_became_active":"2024-09-22T03:52:13.411413+0000","last_became_peered":"2024-09-22T03:52:13.411413+0000","last_unstale":"2024-09-22T04:06:44.477411+0000","last_undegraded":"2024-09-22T04:06:44.477411+0000","last_fullsized":"2024-09-22T04:06:44.477411+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.465248+0000","last_change":"2024-09-22T03:52:13.441353+0000","last_active":"2024-09-22T04:06:44.465248+0000","last_peered":"2024-09-22T04:06:44.465248+0000","last_clean":"2024-09-22T04:06:44.465248+0000","last_became_active":"2024-09-22T03:52:13.399486+0000","last_became_peered":"2024-09-22T03:52:13.399486+0000","last_unstale":"2024-09-22T04:06:44.465248+0000","last_undegraded":"2024-09-22T04:06:44.465248+0000","last_fullsized":"2024-09-22T04:06:44.465248+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.463692+0000","last_change":"2024-09-22T03:52:13.359382+0000","last_active":"2024-09-22T04:06:44.463692+0000","last_peered":"2024-09-22T04:06:44.463692+0000","last_clean":"2024-09-22T04:06:44.463692+0000","last_became_active":"2024-09-22T03:52:13.335952+0000","last_became_peered":"2024-09-22T03:52:13.335952+0000","last_unstale":"2024-09-22T04:06:44.463692+0000","last_undegraded":"2024-09-22T04:06:44.463692+0000","last_fullsized":"2024-09-22T04:06:44.463692+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.498735+0000","last_change":"2024-09-22T03:52:13.445595+0000","last_active":"2024-09-22T04:06:44.498735+0000","last_peered":"2024-09-22T04:06:44.498735+0000","last_clean":"2024-09-22T04:06:44.498735+0000","last_became_active":"2024-09-22T03:52:13.414407+0000","last_became_peered":"2024-09-22T03:52:13.414407+0000","last_unstale":"2024-09-22T04:06:44.498735+0000","last_undegraded":"2024-09-22T04:06:44.498735+0000","last_fullsized":"2024-09-22T04:06:44.498735+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.482567+0000","last_change":"2024-09-22T03:52:13.491741+0000","last_active":"2024-09-22T04:06:44.482567+0000","last_peered":"2024-09-22T04:06:44.482567+0000","last_clean":"2024-09-22T04:06:44.482567+0000","last_became_active":"2024-09-22T03:52:13.431192+0000","last_became_peered":"2024-09-22T03:52:13.431192+0000","last_unstale":"2024-09-22T04:06:44.482567+0000","last_undegraded":"2024-09-22T04:06:44.482567+0000","last_fullsized":"2024-09-22T04:06:44.482567+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.464658+0000","last_change":"2024-09-22T03:52:13.501197+0000","last_active":"2024-09-22T04:06:44.464658+0000","last_peered":"2024-09-22T04:06:44.464658+0000","last_clean":"2024-09-22T04:06:44.464658+0000","last_became_active":"2024-09-22T03:52:13.393363+0000","last_became_peered":"2024-09-22T03:52:13.393363+0000","last_unstale":"2024-09-22T04:06:44.464658+0000","last_undegraded":"2024-09-22T04:06:44.464658+0000","last_fullsized":"2024-09-22T04:06:44.464658+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.490609+0000","last_change":"2024-09-22T03:52:13.454940+0000","last_active":"2024-09-22T04:06:44.490609+0000","last_peered":"2024-09-22T04:06:44.490609+0000","last_clean":"2024-09-22T04:06:44.490609+0000","last_became_active":"2024-09-22T03:52:13.423424+0000","last_became_peered":"2024-09-22T03:52:13.423424+0000","last_unstale":"2024-09-22T04:06:44.490609+0000","last_undegraded":"2024-09-22T04:06:44.490609+0000","last_fullsized":"2024-09-22T04:06:44.490609+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.474681+0000","last_change":"2024-09-22T03:52:13.491666+0000","last_active":"2024-09-22T04:06:44.474681+0000","last_peered":"2024-09-22T04:06:44.474681+0000","last_clean":"2024-09-22T04:06:44.474681+0000","last_became_active":"2024-09-22T03:52:13.404954+0000","last_became_peered":"2024-09-22T03:52:13.404954+0000","last_unstale":"2024-09-22T04:06:44.474681+0000","last_undegraded":"2024-09-22T04:06:44.474681+0000","last_fullsized":"2024-09-22T04:06:44.474681+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"241'7","reported_seq":1497,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.472070+0000","last_change":"2024-09-22T03:52:13.469238+0000","last_active":"2024-09-22T04:06:44.472070+0000","last_peered":"2024-09-22T04:06:44.472070+0000","last_clean":"2024-09-22T04:06:44.472070+0000","last_became_active":"2024-09-22T03:52:13.422437+0000","last_became_peered":"2024-09-22T03:52:13.422437+0000","last_unstale":"2024-09-22T04:06:44.472070+0000","last_undegraded":"2024-09-22T04:06:44.472070+0000","last_fullsized":"2024-09-22T04:06:44.472070+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1484,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.491142+0000","last_change":"2024-09-22T03:52:13.460700+0000","last_active":"2024-09-22T04:06:44.491142+0000","last_peered":"2024-09-22T04:06:44.491142+0000","last_clean":"2024-09-22T04:06:44.491142+0000","last_became_active":"2024-09-22T03:52:13.420437+0000","last_became_peered":"2024-09-22T03:52:13.420437+0000","last_unstale":"2024-09-22T04:06:44.491142+0000","last_undegraded":"2024-09-22T04:06:44.491142+0000","last_fullsized":"2024-09-22T04:06:44.491142+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:52:12.220188+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":2169,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.488571+0000","last_change":"2024-09-22T03:46:57.335250+0000","last_active":"2024-09-22T04:06:44.488571+0000","last_peered":"2024-09-22T04:06:44.488571+0000","last_clean":"2024-09-22T04:06:44.488571+0000","last_became_active":"2024-09-22T03:46:55.378262+0000","last_became_peered":"2024-09-22T03:46:55.378262+0000","last_unstale":"2024-09-22T04:06:44.488571+0000","last_undegraded":"2024-09-22T04:06:44.488571+0000","last_fullsized":"2024-09-22T04:06:44.488571+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2169,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.481104+0000","last_change":"2024-09-22T03:46:57.333626+0000","last_active":"2024-09-22T04:06:44.481104+0000","last_peered":"2024-09-22T04:06:44.481104+0000","last_clean":"2024-09-22T04:06:44.481104+0000","last_became_active":"2024-09-22T03:46:55.378604+0000","last_became_peered":"2024-09-22T03:46:55.378604+0000","last_unstale":"2024-09-22T04:06:44.481104+0000","last_undegraded":"2024-09-22T04:06:44.481104+0000","last_fullsized":"2024-09-22T04:06:44.481104+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2169,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.483589+0000","last_change":"2024-09-22T03:46:57.329704+0000","last_active":"2024-09-22T04:06:44.483589+0000","last_peered":"2024-09-22T04:06:44.483589+0000","last_clean":"2024-09-22T04:06:44.483589+0000","last_became_active":"2024-09-22T03:46:55.370620+0000","last_became_peered":"2024-09-22T03:46:55.370620+0000","last_unstale":"2024-09-22T04:06:44.483589+0000","last_undegraded":"2024-09-22T04:06:44.483589+0000","last_fullsized":"2024-09-22T04:06:44.483589+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2169,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.470822+0000","last_change":"2024-09-22T03:46:57.328548+0000","last_active":"2024-09-22T04:06:44.470822+0000","last_peered":"2024-09-22T04:06:44.470822+0000","last_clean":"2024-09-22T04:06:44.470822+0000","last_became_active":"2024-09-22T03:46:55.358980+0000","last_became_peered":"2024-09-22T03:46:55.358980+0000","last_unstale":"2024-09-22T04:06:44.470822+0000","last_undegraded":"2024-09-22T04:06:44.470822+0000","last_fullsized":"2024-09-22T04:06:44.470822+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":2169,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.492255+0000","last_change":"2024-09-22T03:46:57.331270+0000","last_active":"2024-09-22T04:06:44.492255+0000","last_peered":"2024-09-22T04:06:44.492255+0000","last_clean":"2024-09-22T04:06:44.492255+0000","last_became_active":"2024-09-22T03:46:55.383021+0000","last_became_peered":"2024-09-22T03:46:55.383021+0000","last_unstale":"2024-09-22T04:06:44.492255+0000","last_undegraded":"2024-09-22T04:06:44.492255+0000","last_fullsized":"2024-09-22T04:06:44.492255+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2169,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.466290+0000","last_change":"2024-09-22T03:46:57.329346+0000","last_active":"2024-09-22T04:06:44.466290+0000","last_peered":"2024-09-22T04:06:44.466290+0000","last_clean":"2024-09-22T04:06:44.466290+0000","last_became_active":"2024-09-22T03:46:55.358542+0000","last_became_peered":"2024-09-22T03:46:55.358542+0000","last_unstale":"2024-09-22T04:06:44.466290+0000","last_undegraded":"2024-09-22T04:06:44.466290+0000","last_fullsized":"2024-09-22T04:06:44.466290+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2167,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.478791+0000","last_change":"2024-09-22T03:46:57.332056+0000","last_active":"2024-09-22T04:06:44.478791+0000","last_peered":"2024-09-22T04:06:44.478791+0000","last_clean":"2024-09-22T04:06:44.478791+0000","last_became_active":"2024-09-22T03:46:55.412604+0000","last_became_peered":"2024-09-22T03:46:55.412604+0000","last_unstale":"2024-09-22T04:06:44.478791+0000","last_undegraded":"2024-09-22T04:06:44.478791+0000","last_fullsized":"2024-09-22T04:06:44.478791+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":2170,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.487146+0000","last_change":"2024-09-22T03:46:57.335158+0000","last_active":"2024-09-22T04:06:44.487146+0000","last_peered":"2024-09-22T04:06:44.487146+0000","last_clean":"2024-09-22T04:06:44.487146+0000","last_became_active":"2024-09-22T03:46:55.387128+0000","last_became_peered":"2024-09-22T03:46:55.387128+0000","last_unstale":"2024-09-22T04:06:44.487146+0000","last_undegraded":"2024-09-22T04:06:44.487146+0000","last_fullsized":"2024-09-22T04:06:44.487146+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:54.283073+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":2241,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:44.458607+0000","last_change":"2024-09-22T03:46:53.370744+0000","last_active":"2024-09-22T04:06:44.458607+0000","last_peered":"2024-09-22T04:06:44.458607+0000","last_clean":"2024-09-22T04:06:44.458607+0000","last_became_active":"2024-09-22T03:46:53.349788+0000","last_became_peered":"2024-09-22T03:46:53.349788+0000","last_unstale":"2024-09-22T04:06:44.458607+0000","last_undegraded":"2024-09-22T04:06:44.458607+0000","last_fullsized":"2024-09-22T04:06:44.458607+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T03:46:52.275666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T03:46:52.275666+0000","last_clean_scrub_stamp":"2024-09-22T03:46:52.275666+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214194618368,"internally_reserved":0,"allocated":16875520,"data_stored":16875520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214194618368,"internally_reserved":0,"allocated":16875520,"data_stored":16875520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53548302336,"internally_reserved":0,"allocated":4571136,"data_stored":4571136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706858,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":4464,"kb_used_data":4464,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52293264,"statfs":{"total":53552873472,"available":53548302336,"internally_reserved":0,"allocated":4571136,"data_stored":4571136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706858,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":3596,"kb_used_data":3596,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52294132,"statfs":{"total":53552873472,"available":53549191168,"internally_reserved":0,"allocated":3682304,"data_stored":3682304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706858,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":3568,"kb_used_data":3568,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52294160,"statfs":{"total":53552873472,"available":53549219840,"internally_reserved":0,"allocated":3653632,"data_stored":3653632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706857,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":5716,"kb_used_data":5716,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52292012,"statfs":{"total":53552873472,"available":53547020288,"internally_reserved":0,"allocated":5853184,"data_stored":5853184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53548302336,"internally_reserved":0,"allocated":4571136,"data_stored":4571136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53547905024,"internally_reserved":0,"allocated":4968448,"data_stored":4968448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53549219840,"internally_reserved":0,"allocated":3653632,"data_stored":3653632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53549191168,"internally_reserved":0,"allocated":3682304,"data_stored":3682304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53548302336,"internally_reserved":0,"allocated":4571136,"data_stored":4571136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":0,"total":53552873472,"available":53547905024,"internally_reserved":0,"allocated":4968448,"data_stored":4968448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":1,"total":53552873472,"available":53549219840,"internally_reserved":0,"allocated":3653632,"data_stored":3653632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":2,"total":53552873472,"available":53549191168,"internally_reserved":0,"allocated":3682304,"data_stored":3682304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":3,"total":53552873472,"available":53548302336,"internally_reserved":0,"allocated":4571136,"data_stored":4571136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T04:06:49.844 INFO:tasks.ceph:pgid 75.1f last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.844 INFO:tasks.ceph:pgid 75.1e last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.844 INFO:tasks.ceph:pgid 75.1d last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.844 INFO:tasks.ceph:pgid 75.1c last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.844 INFO:tasks.ceph:pgid 75.1b last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.845 INFO:tasks.ceph:pgid 75.1a last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.845 INFO:tasks.ceph:pgid 75.19 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.845 INFO:tasks.ceph:pgid 75.18 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.845 INFO:tasks.ceph:pgid 75.17 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.845 INFO:tasks.ceph:pgid 75.16 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.845 INFO:tasks.ceph:pgid 75.15 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.845 INFO:tasks.ceph:pgid 75.14 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.845 INFO:tasks.ceph:pgid 75.13 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.845 INFO:tasks.ceph:pgid 75.12 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.846 INFO:tasks.ceph:pgid 75.11 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.846 INFO:tasks.ceph:pgid 75.10 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.846 INFO:tasks.ceph:pgid 75.f last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.846 INFO:tasks.ceph:pgid 75.e last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.846 INFO:tasks.ceph:pgid 75.d last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.846 INFO:tasks.ceph:pgid 75.c last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.846 INFO:tasks.ceph:pgid 75.b last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.846 INFO:tasks.ceph:pgid 75.a last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.846 INFO:tasks.ceph:pgid 75.9 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.846 INFO:tasks.ceph:pgid 75.8 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.847 INFO:tasks.ceph:pgid 75.7 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.847 INFO:tasks.ceph:pgid 75.6 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.847 INFO:tasks.ceph:pgid 75.5 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.847 INFO:tasks.ceph:pgid 75.4 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.847 INFO:tasks.ceph:pgid 75.3 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.847 INFO:tasks.ceph:pgid 75.2 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.847 INFO:tasks.ceph:pgid 75.1 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.847 INFO:tasks.ceph:pgid 75.0 last_scrub_stamp 2024-09-22T03:52:12.220188+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=52, tm_sec=12, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.847 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-22T03:46:54.283073+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=46, tm_sec=54, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.848 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-22T03:46:54.283073+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=46, tm_sec=54, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.848 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-22T03:46:54.283073+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=46, tm_sec=54, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.848 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-22T03:46:54.283073+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=46, tm_sec=54, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.848 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-22T03:46:54.283073+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=46, tm_sec=54, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.848 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-22T03:46:54.283073+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=46, tm_sec=54, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.848 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-22T03:46:54.283073+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=46, tm_sec=54, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.848 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-22T03:46:54.283073+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=46, tm_sec=54, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.848 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-22T03:46:52.275666+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=3, tm_min=46, tm_sec=52, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=4, tm_min=6, tm_sec=46, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T04:06:49.848 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-22T04:07:09.849 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T04:07:09.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.961+0000 7fe9c4c8e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:07:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.966+0000 7fe9c4c8e640 1 Processor -- start 2024-09-22T04:07:09.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.966+0000 7fe9c4c8e640 1 -- start start 2024-09-22T04:07:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.968+0000 7fe9c4c8e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9c006eb50 0x7fe9c01334c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:07:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.968+0000 7fe9c4c8e640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9c01329d0 con 0x7fe9c006fb10 2024-09-22T04:07:09.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.968+0000 7fe9c4c8e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe9c00702e0 con 0x7fe9c006eb50 2024-09-22T04:07:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.968+0000 7fe9be59c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe9c006fb10 0x7fe9c0132db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37342/0 (socket says 172.21.15.3:37342) 2024-09-22T04:07:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.968+0000 7fe9be59c640 1 -- 172.21.15.3:0/1078442761 learned_addr learned my addr 172.21.15.3:0/1078442761 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T04:07:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.968+0000 7fe9bdd9b640 1 --2- 172.21.15.3:0/1078442761 >> v2:172.21.15.3:3300/0 conn(0x7fe9c006eb50 0x7fe9c01334c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:07:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.968+0000 7fe9bd59a640 1 -- 172.21.15.3:0/1078442761 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3439422767 0 0) 0x7fe9c01329d0 con 0x7fe9c006fb10 2024-09-22T04:07:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.968+0000 7fe9bd59a640 1 -- 172.21.15.3:0/1078442761 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9a8005790 con 0x7fe9c006fb10 2024-09-22T04:07:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.969+0000 7fe9bdd9b640 1 -- 172.21.15.3:0/1078442761 >> v1:172.21.15.3:6789/0 conn(0x7fe9c006fb10 legacy=0x7fe9c0132db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:07:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.969+0000 7fe9bdd9b640 1 -- 172.21.15.3:0/1078442761 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c006f0d0 con 0x7fe9c006eb50 2024-09-22T04:07:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.969+0000 7fe9bdd9b640 1 --2- 172.21.15.3:0/1078442761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c006eb50 0x7fe9c01334c0 crc :-1 s=READY pgs=13042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f44941641c7a4a server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:07:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.969+0000 7fe9bd59a640 1 -- 172.21.15.3:0/1078442761 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe9b4009d90 con 0x7fe9c006eb50 2024-09-22T04:07:09.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.969+0000 7fe9bd59a640 1 -- 172.21.15.3:0/1078442761 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe9b400e230 con 0x7fe9c006eb50 2024-09-22T04:07:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.969+0000 7fe9bd59a640 1 -- 172.21.15.3:0/1078442761 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe9b400e4d0 con 0x7fe9c006eb50 2024-09-22T04:07:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.970+0000 7fe9c4c8e640 1 -- 172.21.15.3:0/1078442761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c006eb50 msgr2=0x7fe9c01334c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:07:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.970+0000 7fe9c4c8e640 1 --2- 172.21.15.3:0/1078442761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c006eb50 0x7fe9c01334c0 crc :-1 s=READY pgs=13042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:07:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.970+0000 7fe9c4c8e640 1 -- 172.21.15.3:0/1078442761 shutdown_connections 2024-09-22T04:07:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.970+0000 7fe9c4c8e640 1 --2- 172.21.15.3:0/1078442761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c006eb50 0x7fe9c01334c0 unknown :-1 s=CLOSED pgs=13042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:07:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.970+0000 7fe9c4c8e640 1 -- 172.21.15.3:0/1078442761 >> 172.21.15.3:0/1078442761 conn(0x7fe9c00a1bd0 msgr2=0x7fe9c00a2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:07:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.970+0000 7fe9c4c8e640 1 -- 172.21.15.3:0/1078442761 shutdown_connections 2024-09-22T04:07:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.970+0000 7fe9c4c8e640 1 -- 172.21.15.3:0/1078442761 wait complete. 2024-09-22T04:07:09.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.970+0000 7fe9c4c8e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:07:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.972+0000 7fe9c4c8e640 1 Processor -- start 2024-09-22T04:07:09.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.972+0000 7fe9c4c8e640 1 -- start start 2024-09-22T04:07:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.973+0000 7fe9c4c8e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c006eb50 0x7fe9c011c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:07:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.973+0000 7fe9c4c8e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c00a05a0 con 0x7fe9c006eb50 2024-09-22T04:07:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.973+0000 7fe9be59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c006eb50 0x7fe9c011c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:07:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.973+0000 7fe9be59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c006eb50 0x7fe9c011c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36340/0 (socket says 172.21.15.3:36340) 2024-09-22T04:07:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.974+0000 7fe9be59c640 1 -- 172.21.15.3:0/2970175133 learned_addr learned my addr 172.21.15.3:0/2970175133 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:07:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.974+0000 7fe9be59c640 1 -- 172.21.15.3:0/2970175133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c011f570 con 0x7fe9c006eb50 2024-09-22T04:07:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.974+0000 7fe9be59c640 1 --2- 172.21.15.3:0/2970175133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c006eb50 0x7fe9c011c380 crc :-1 s=READY pgs=13043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:07:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.974+0000 7fe9aeffd640 1 -- 172.21.15.3:0/2970175133 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe9b00070b0 con 0x7fe9c006eb50 2024-09-22T04:07:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.974+0000 7fe9aeffd640 1 -- 172.21.15.3:0/2970175133 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fe9b0007210 con 0x7fe9c006eb50 2024-09-22T04:07:09.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.974+0000 7fe9c4c8e640 1 -- 172.21.15.3:0/2970175133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c011e5f0 con 0x7fe9c006eb50 2024-09-22T04:07:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.974+0000 7fe9aeffd640 1 -- 172.21.15.3:0/2970175133 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fe9b00074b0 con 0x7fe9c006eb50 2024-09-22T04:07:09.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.974+0000 7fe9c4c8e640 1 -- 172.21.15.3:0/2970175133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c011fe40 con 0x7fe9c006eb50 2024-09-22T04:07:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.975+0000 7fe9c4c8e640 1 -- 172.21.15.3:0/2970175133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9c0132db0 con 0x7fe9c006eb50 2024-09-22T04:07:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.975+0000 7fe9aeffd640 1 -- 172.21.15.3:0/2970175133 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fe9b00076a0 con 0x7fe9c006eb50 2024-09-22T04:07:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.976+0000 7fe9aeffd640 1 --2- 172.21.15.3:0/2970175133 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe9a003ea80 0x7fe9a0040f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:07:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.976+0000 7fe9aeffd640 1 -- 172.21.15.3:0/2970175133 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(733..733 src has 683..733) ==== 3431+0+0 (crc 0 0 0) 0x7fe9b0050560 con 0x7fe9c006eb50 2024-09-22T04:07:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.976+0000 7fe9bdd9b640 1 --2- 172.21.15.3:0/2970175133 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe9a003ea80 0x7fe9a0040f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:07:09.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.977+0000 7fe9bdd9b640 1 --2- 172.21.15.3:0/2970175133 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe9a003ea80 0x7fe9a0040f90 crc :-1 s=READY pgs=3882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:07:09.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:09.981+0000 7fe9aeffd640 1 -- 172.21.15.3:0/2970175133 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7fe9b0009830 con 0x7fe9c006eb50 2024-09-22T04:07:10.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.164+0000 7fe9c4c8e640 1 -- 172.21.15.3:0/2970175133 --> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe9c011f400 con 0x7fe9a003ea80 2024-09-22T04:07:10.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.167+0000 7fe9aeffd640 1 -- 172.21.15.3:0/2970175133 <== mgr.4099 v2:172.21.15.123:6800/551784226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+109422 (crc 0 0 0) 0x7fe9c011f400 con 0x7fe9a003ea80 2024-09-22T04:07:10.169 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-22T04:07:10.170 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-22T04:07:10.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.169+0000 7fe9c4c8e640 1 -- 172.21.15.3:0/2970175133 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe9a003ea80 msgr2=0x7fe9a0040f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:07:10.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.169+0000 7fe9c4c8e640 1 --2- 172.21.15.3:0/2970175133 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe9a003ea80 0x7fe9a0040f90 crc :-1 s=READY pgs=3882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:07:10.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.169+0000 7fe9c4c8e640 1 -- 172.21.15.3:0/2970175133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c006eb50 msgr2=0x7fe9c011c380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:07:10.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.169+0000 7fe9c4c8e640 1 --2- 172.21.15.3:0/2970175133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c006eb50 0x7fe9c011c380 crc :-1 s=READY pgs=13043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:07:10.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.169+0000 7fe9c4c8e640 1 -- 172.21.15.3:0/2970175133 shutdown_connections 2024-09-22T04:07:10.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.169+0000 7fe9c4c8e640 1 --2- 172.21.15.3:0/2970175133 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fe9a003ea80 0x7fe9a0040f90 unknown :-1 s=CLOSED pgs=3882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:07:10.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.169+0000 7fe9c4c8e640 1 --2- 172.21.15.3:0/2970175133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c006eb50 0x7fe9c011c380 unknown :-1 s=CLOSED pgs=13043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:07:10.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.169+0000 7fe9c4c8e640 1 -- 172.21.15.3:0/2970175133 >> 172.21.15.3:0/2970175133 conn(0x7fe9c00a1bd0 msgr2=0x7fe9c009d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:07:10.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.169+0000 7fe9c4c8e640 1 -- 172.21.15.3:0/2970175133 shutdown_connections 2024-09-22T04:07:10.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.169+0000 7fe9c4c8e640 1 -- 172.21.15.3:0/2970175133 wait complete. 2024-09-22T04:07:10.181 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":1337,"stamp":"2024-09-22T04:07:09.848049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590392,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":52,"num_write":88,"num_write_kb":1115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62,"ondisk_log_size":62,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209190912,"kb_used":16480,"kb_used_data":16480,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":209174432,"statfs":{"total":214211493888,"available":214194618368,"internally_reserved":0,"allocated":16875520,"data_stored":16875520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003354"},"pg_stats":[{"pgid":"75.1f","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.517651+0000","last_change":"2024-09-22T04:06:48.555379+0000","last_active":"2024-09-22T04:06:48.517651+0000","last_peered":"2024-09-22T04:06:48.517651+0000","last_clean":"2024-09-22T04:06:48.517651+0000","last_became_active":"2024-09-22T03:52:13.436104+0000","last_became_peered":"2024-09-22T03:52:13.436104+0000","last_unstale":"2024-09-22T04:06:48.517651+0000","last_undegraded":"2024-09-22T04:06:48.517651+0000","last_fullsized":"2024-09-22T04:06:48.517651+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.555200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.555200+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1e","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.019298+0000","last_change":"2024-09-22T04:06:48.066958+0000","last_active":"2024-09-22T04:06:48.019298+0000","last_peered":"2024-09-22T04:06:48.019298+0000","last_clean":"2024-09-22T04:06:48.019298+0000","last_became_active":"2024-09-22T03:52:13.485941+0000","last_became_peered":"2024-09-22T03:52:13.485941+0000","last_unstale":"2024-09-22T04:06:48.019298+0000","last_undegraded":"2024-09-22T04:06:48.019298+0000","last_fullsized":"2024-09-22T04:06:48.019298+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.066773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.066773+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1d","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.016449+0000","last_change":"2024-09-22T04:06:49.048017+0000","last_active":"2024-09-22T04:06:49.016449+0000","last_peered":"2024-09-22T04:06:49.016449+0000","last_clean":"2024-09-22T04:06:49.016449+0000","last_became_active":"2024-09-22T03:52:13.435054+0000","last_became_peered":"2024-09-22T03:52:13.435054+0000","last_unstale":"2024-09-22T04:06:49.016449+0000","last_undegraded":"2024-09-22T04:06:49.016449+0000","last_fullsized":"2024-09-22T04:06:49.016449+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:49.047813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.047813+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.1c","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.506525+0000","last_change":"2024-09-22T04:06:49.538445+0000","last_active":"2024-09-22T04:06:49.506525+0000","last_peered":"2024-09-22T04:06:49.506525+0000","last_clean":"2024-09-22T04:06:49.506525+0000","last_became_active":"2024-09-22T03:52:13.424125+0000","last_became_peered":"2024-09-22T03:52:13.424125+0000","last_unstale":"2024-09-22T04:06:49.506525+0000","last_undegraded":"2024-09-22T04:06:49.506525+0000","last_fullsized":"2024-09-22T04:06:49.506525+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:49.538152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.538152+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.1b","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.520102+0000","last_change":"2024-09-22T04:06:48.562558+0000","last_active":"2024-09-22T04:06:48.520102+0000","last_peered":"2024-09-22T04:06:48.520102+0000","last_clean":"2024-09-22T04:06:48.520102+0000","last_became_active":"2024-09-22T03:52:13.612276+0000","last_became_peered":"2024-09-22T03:52:13.612276+0000","last_unstale":"2024-09-22T04:06:48.520102+0000","last_undegraded":"2024-09-22T04:06:48.520102+0000","last_fullsized":"2024-09-22T04:06:48.520102+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.562390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.562390+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.1a","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.016757+0000","last_change":"2024-09-22T04:06:48.065674+0000","last_active":"2024-09-22T04:06:48.016757+0000","last_peered":"2024-09-22T04:06:48.016757+0000","last_clean":"2024-09-22T04:06:48.016757+0000","last_became_active":"2024-09-22T03:52:13.584449+0000","last_became_peered":"2024-09-22T03:52:13.584449+0000","last_unstale":"2024-09-22T04:06:48.016757+0000","last_undegraded":"2024-09-22T04:06:48.016757+0000","last_fullsized":"2024-09-22T04:06:48.016757+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.065471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.065471+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.19","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.018719+0000","last_change":"2024-09-22T04:06:48.058215+0000","last_active":"2024-09-22T04:06:48.018719+0000","last_peered":"2024-09-22T04:06:48.018719+0000","last_clean":"2024-09-22T04:06:48.018719+0000","last_became_active":"2024-09-22T03:52:13.438920+0000","last_became_peered":"2024-09-22T03:52:13.438920+0000","last_unstale":"2024-09-22T04:06:48.018719+0000","last_undegraded":"2024-09-22T04:06:48.018719+0000","last_fullsized":"2024-09-22T04:06:48.018719+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.058026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.058026+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.18","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.017546+0000","last_change":"2024-09-22T04:06:48.059879+0000","last_active":"2024-09-22T04:06:48.017546+0000","last_peered":"2024-09-22T04:06:48.017546+0000","last_clean":"2024-09-22T04:06:48.017546+0000","last_became_active":"2024-09-22T03:52:13.424399+0000","last_became_peered":"2024-09-22T03:52:13.424399+0000","last_unstale":"2024-09-22T04:06:48.017546+0000","last_undegraded":"2024-09-22T04:06:48.017546+0000","last_fullsized":"2024-09-22T04:06:48.017546+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.059701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.059701+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.17","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.506311+0000","last_change":"2024-09-22T04:06:49.563787+0000","last_active":"2024-09-22T04:06:49.506311+0000","last_peered":"2024-09-22T04:06:49.506311+0000","last_clean":"2024-09-22T04:06:49.506311+0000","last_became_active":"2024-09-22T03:52:13.367950+0000","last_became_peered":"2024-09-22T03:52:13.367950+0000","last_unstale":"2024-09-22T04:06:49.506311+0000","last_undegraded":"2024-09-22T04:06:49.506311+0000","last_fullsized":"2024-09-22T04:06:49.506311+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:49.563610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.563610+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.16","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.013672+0000","last_change":"2024-09-22T04:06:49.036316+0000","last_active":"2024-09-22T04:06:49.013672+0000","last_peered":"2024-09-22T04:06:49.013672+0000","last_clean":"2024-09-22T04:06:49.013672+0000","last_became_active":"2024-09-22T03:52:13.370886+0000","last_became_peered":"2024-09-22T03:52:13.370886+0000","last_unstale":"2024-09-22T04:06:49.013672+0000","last_undegraded":"2024-09-22T04:06:49.013672+0000","last_fullsized":"2024-09-22T04:06:49.013672+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:49.036114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.036114+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.15","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.506517+0000","last_change":"2024-09-22T04:06:49.550536+0000","last_active":"2024-09-22T04:06:49.506517+0000","last_peered":"2024-09-22T04:06:49.506517+0000","last_clean":"2024-09-22T04:06:49.506517+0000","last_became_active":"2024-09-22T03:52:13.477618+0000","last_became_peered":"2024-09-22T03:52:13.477618+0000","last_unstale":"2024-09-22T04:06:49.506517+0000","last_undegraded":"2024-09-22T04:06:49.506517+0000","last_fullsized":"2024-09-22T04:06:49.506517+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:49.550364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.550364+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.14","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.012367+0000","last_change":"2024-09-22T04:06:48.056470+0000","last_active":"2024-09-22T04:06:48.012367+0000","last_peered":"2024-09-22T04:06:48.012367+0000","last_clean":"2024-09-22T04:06:48.012367+0000","last_became_active":"2024-09-22T03:52:13.354826+0000","last_became_peered":"2024-09-22T03:52:13.354826+0000","last_unstale":"2024-09-22T04:06:48.012367+0000","last_undegraded":"2024-09-22T04:06:48.012367+0000","last_fullsized":"2024-09-22T04:06:48.012367+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.056288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.056288+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.13","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.502987+0000","last_change":"2024-09-22T04:06:49.555008+0000","last_active":"2024-09-22T04:06:49.502987+0000","last_peered":"2024-09-22T04:06:49.502987+0000","last_clean":"2024-09-22T04:06:49.502987+0000","last_became_active":"2024-09-22T03:52:13.405184+0000","last_became_peered":"2024-09-22T03:52:13.405184+0000","last_unstale":"2024-09-22T04:06:49.502987+0000","last_undegraded":"2024-09-22T04:06:49.502987+0000","last_fullsized":"2024-09-22T04:06:49.502987+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:49.554803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.554803+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.12","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.502971+0000","last_change":"2024-09-22T04:06:49.538754+0000","last_active":"2024-09-22T04:06:49.502971+0000","last_peered":"2024-09-22T04:06:49.502971+0000","last_clean":"2024-09-22T04:06:49.502971+0000","last_became_active":"2024-09-22T03:52:13.533286+0000","last_became_peered":"2024-09-22T03:52:13.533286+0000","last_unstale":"2024-09-22T04:06:49.502971+0000","last_undegraded":"2024-09-22T04:06:49.502971+0000","last_fullsized":"2024-09-22T04:06:49.502971+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:49.538579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.538579+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.11","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.015490+0000","last_change":"2024-09-22T04:06:48.063432+0000","last_active":"2024-09-22T04:06:48.015490+0000","last_peered":"2024-09-22T04:06:48.015490+0000","last_clean":"2024-09-22T04:06:48.015490+0000","last_became_active":"2024-09-22T03:52:13.531887+0000","last_became_peered":"2024-09-22T03:52:13.531887+0000","last_unstale":"2024-09-22T04:06:48.015490+0000","last_undegraded":"2024-09-22T04:06:48.015490+0000","last_fullsized":"2024-09-22T04:06:48.015490+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.063246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.063246+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.10","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.013275+0000","last_change":"2024-09-22T04:06:48.061116+0000","last_active":"2024-09-22T04:06:48.013275+0000","last_peered":"2024-09-22T04:06:48.013275+0000","last_clean":"2024-09-22T04:06:48.013275+0000","last_became_active":"2024-09-22T03:52:13.562231+0000","last_became_peered":"2024-09-22T03:52:13.562231+0000","last_unstale":"2024-09-22T04:06:48.013275+0000","last_undegraded":"2024-09-22T04:06:48.013275+0000","last_fullsized":"2024-09-22T04:06:48.013275+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.060934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.060934+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.f","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.011022+0000","last_change":"2024-09-22T04:06:48.061824+0000","last_active":"2024-09-22T04:06:48.011022+0000","last_peered":"2024-09-22T04:06:48.011022+0000","last_clean":"2024-09-22T04:06:48.011022+0000","last_became_active":"2024-09-22T03:52:13.496766+0000","last_became_peered":"2024-09-22T03:52:13.496766+0000","last_unstale":"2024-09-22T04:06:48.011022+0000","last_undegraded":"2024-09-22T04:06:48.011022+0000","last_fullsized":"2024-09-22T04:06:48.011022+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.061649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.061649+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.e","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.500795+0000","last_change":"2024-09-22T04:06:49.551581+0000","last_active":"2024-09-22T04:06:49.500795+0000","last_peered":"2024-09-22T04:06:49.500795+0000","last_clean":"2024-09-22T04:06:49.500795+0000","last_became_active":"2024-09-22T03:52:13.496886+0000","last_became_peered":"2024-09-22T03:52:13.496886+0000","last_unstale":"2024-09-22T04:06:49.500795+0000","last_undegraded":"2024-09-22T04:06:49.500795+0000","last_fullsized":"2024-09-22T04:06:49.500795+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:49.551413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.551413+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.d","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.016158+0000","last_change":"2024-09-22T04:06:49.042559+0000","last_active":"2024-09-22T04:06:49.016158+0000","last_peered":"2024-09-22T04:06:49.016158+0000","last_clean":"2024-09-22T04:06:49.016158+0000","last_became_active":"2024-09-22T03:52:13.412673+0000","last_became_peered":"2024-09-22T03:52:13.412673+0000","last_unstale":"2024-09-22T04:06:49.016158+0000","last_undegraded":"2024-09-22T04:06:49.016158+0000","last_fullsized":"2024-09-22T04:06:49.016158+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:49.042275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.042275+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.c","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.518223+0000","last_change":"2024-09-22T04:06:48.548465+0000","last_active":"2024-09-22T04:06:48.518223+0000","last_peered":"2024-09-22T04:06:48.518223+0000","last_clean":"2024-09-22T04:06:48.518223+0000","last_became_active":"2024-09-22T03:52:13.441995+0000","last_became_peered":"2024-09-22T03:52:13.441995+0000","last_unstale":"2024-09-22T04:06:48.518223+0000","last_undegraded":"2024-09-22T04:06:48.518223+0000","last_fullsized":"2024-09-22T04:06:48.518223+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.548283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.548283+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.b","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.514448+0000","last_change":"2024-09-22T04:06:48.558371+0000","last_active":"2024-09-22T04:06:48.514448+0000","last_peered":"2024-09-22T04:06:48.514448+0000","last_clean":"2024-09-22T04:06:48.514448+0000","last_became_active":"2024-09-22T03:52:13.657594+0000","last_became_peered":"2024-09-22T03:52:13.657594+0000","last_unstale":"2024-09-22T04:06:48.514448+0000","last_undegraded":"2024-09-22T04:06:48.514448+0000","last_fullsized":"2024-09-22T04:06:48.514448+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.558194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.558194+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.a","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.518889+0000","last_change":"2024-09-22T04:06:48.558266+0000","last_active":"2024-09-22T04:06:48.518889+0000","last_peered":"2024-09-22T04:06:48.518889+0000","last_clean":"2024-09-22T04:06:48.518889+0000","last_became_active":"2024-09-22T03:52:13.435529+0000","last_became_peered":"2024-09-22T03:52:13.435529+0000","last_unstale":"2024-09-22T04:06:48.518889+0000","last_undegraded":"2024-09-22T04:06:48.518889+0000","last_fullsized":"2024-09-22T04:06:48.518889+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.558057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.558057+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.9","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.013381+0000","last_change":"2024-09-22T04:06:49.044948+0000","last_active":"2024-09-22T04:06:49.013381+0000","last_peered":"2024-09-22T04:06:49.013381+0000","last_clean":"2024-09-22T04:06:49.013381+0000","last_became_active":"2024-09-22T03:52:13.411413+0000","last_became_peered":"2024-09-22T03:52:13.411413+0000","last_unstale":"2024-09-22T04:06:49.013381+0000","last_undegraded":"2024-09-22T04:06:49.013381+0000","last_fullsized":"2024-09-22T04:06:49.013381+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:49.044755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.044755+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"75.8","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.499909+0000","last_change":"2024-09-22T04:06:49.560192+0000","last_active":"2024-09-22T04:06:49.499909+0000","last_peered":"2024-09-22T04:06:49.499909+0000","last_clean":"2024-09-22T04:06:49.499909+0000","last_became_active":"2024-09-22T03:52:13.399486+0000","last_became_peered":"2024-09-22T03:52:13.399486+0000","last_unstale":"2024-09-22T04:06:49.499909+0000","last_undegraded":"2024-09-22T04:06:49.499909+0000","last_fullsized":"2024-09-22T04:06:49.499909+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:49.560016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.560016+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.7","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.497627+0000","last_change":"2024-09-22T04:06:49.557496+0000","last_active":"2024-09-22T04:06:49.497627+0000","last_peered":"2024-09-22T04:06:49.497627+0000","last_clean":"2024-09-22T04:06:49.497627+0000","last_became_active":"2024-09-22T03:52:13.335952+0000","last_became_peered":"2024-09-22T03:52:13.335952+0000","last_unstale":"2024-09-22T04:06:49.497627+0000","last_undegraded":"2024-09-22T04:06:49.497627+0000","last_fullsized":"2024-09-22T04:06:49.497627+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:49.557336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.557336+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.6","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.012774+0000","last_change":"2024-09-22T04:06:48.053324+0000","last_active":"2024-09-22T04:06:48.012774+0000","last_peered":"2024-09-22T04:06:48.012774+0000","last_clean":"2024-09-22T04:06:48.012774+0000","last_became_active":"2024-09-22T03:52:13.414407+0000","last_became_peered":"2024-09-22T03:52:13.414407+0000","last_unstale":"2024-09-22T04:06:48.012774+0000","last_undegraded":"2024-09-22T04:06:48.012774+0000","last_fullsized":"2024-09-22T04:06:48.012774+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.053036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.053036+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.5","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.497679+0000","last_change":"2024-09-22T04:06:49.523522+0000","last_active":"2024-09-22T04:06:49.497679+0000","last_peered":"2024-09-22T04:06:49.497679+0000","last_clean":"2024-09-22T04:06:49.497679+0000","last_became_active":"2024-09-22T03:52:13.431192+0000","last_became_peered":"2024-09-22T03:52:13.431192+0000","last_unstale":"2024-09-22T04:06:49.497679+0000","last_undegraded":"2024-09-22T04:06:49.497679+0000","last_fullsized":"2024-09-22T04:06:49.497679+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:49.523313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.523313+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"75.4","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.008519+0000","last_change":"2024-09-22T04:06:48.053002+0000","last_active":"2024-09-22T04:06:48.008519+0000","last_peered":"2024-09-22T04:06:48.008519+0000","last_clean":"2024-09-22T04:06:48.008519+0000","last_became_active":"2024-09-22T03:52:13.393363+0000","last_became_peered":"2024-09-22T03:52:13.393363+0000","last_unstale":"2024-09-22T04:06:48.008519+0000","last_undegraded":"2024-09-22T04:06:48.008519+0000","last_fullsized":"2024-09-22T04:06:48.008519+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.052666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.052666+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.3","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.010247+0000","last_change":"2024-09-22T04:06:48.045711+0000","last_active":"2024-09-22T04:06:48.010247+0000","last_peered":"2024-09-22T04:06:48.010247+0000","last_clean":"2024-09-22T04:06:48.010247+0000","last_became_active":"2024-09-22T03:52:13.423424+0000","last_became_peered":"2024-09-22T03:52:13.423424+0000","last_unstale":"2024-09-22T04:06:48.010247+0000","last_undegraded":"2024-09-22T04:06:48.010247+0000","last_fullsized":"2024-09-22T04:06:48.010247+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.045517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.045517+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.2","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.006242+0000","last_change":"2024-09-22T04:06:48.046564+0000","last_active":"2024-09-22T04:06:48.006242+0000","last_peered":"2024-09-22T04:06:48.006242+0000","last_clean":"2024-09-22T04:06:48.006242+0000","last_became_active":"2024-09-22T03:52:13.404954+0000","last_became_peered":"2024-09-22T03:52:13.404954+0000","last_unstale":"2024-09-22T04:06:48.006242+0000","last_undegraded":"2024-09-22T04:06:48.006242+0000","last_fullsized":"2024-09-22T04:06:48.006242+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.046384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.046384+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"75.1","version":"241'7","reported_seq":1499,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.514063+0000","last_change":"2024-09-22T04:06:48.565546+0000","last_active":"2024-09-22T04:06:48.514063+0000","last_peered":"2024-09-22T04:06:48.514063+0000","last_clean":"2024-09-22T04:06:48.514063+0000","last_became_active":"2024-09-22T03:52:13.422437+0000","last_became_peered":"2024-09-22T03:52:13.422437+0000","last_unstale":"2024-09-22T04:06:48.514063+0000","last_undegraded":"2024-09-22T04:06:48.514063+0000","last_fullsized":"2024-09-22T04:06:48.514063+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"241'7","last_scrub_stamp":"2024-09-22T04:06:48.565332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.565332+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"75.0","version":"0'0","reported_seq":1486,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.013392+0000","last_change":"2024-09-22T04:06:49.037513+0000","last_active":"2024-09-22T04:06:49.013392+0000","last_peered":"2024-09-22T04:06:49.013392+0000","last_clean":"2024-09-22T04:06:49.013392+0000","last_became_active":"2024-09-22T03:52:13.420437+0000","last_became_peered":"2024-09-22T03:52:13.420437+0000","last_unstale":"2024-09-22T04:06:49.013392+0000","last_undegraded":"2024-09-22T04:06:49.013392+0000","last_fullsized":"2024-09-22T04:06:49.013392+0000","mapping_epoch":240,"log_start":"0'0","ondisk_log_start":"0'0","created":240,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:49.037215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.037215+0000","last_clean_scrub_stamp":"2024-09-22T03:52:12.220188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":2171,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.513357+0000","last_change":"2024-09-22T04:06:48.536090+0000","last_active":"2024-09-22T04:06:48.513357+0000","last_peered":"2024-09-22T04:06:48.513357+0000","last_clean":"2024-09-22T04:06:48.513357+0000","last_became_active":"2024-09-22T03:46:55.378262+0000","last_became_peered":"2024-09-22T03:46:55.378262+0000","last_unstale":"2024-09-22T04:06:48.513357+0000","last_undegraded":"2024-09-22T04:06:48.513357+0000","last_fullsized":"2024-09-22T04:06:48.513357+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.535912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.535912+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2171,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.012301+0000","last_change":"2024-09-22T04:06:49.040536+0000","last_active":"2024-09-22T04:06:49.012301+0000","last_peered":"2024-09-22T04:06:49.012301+0000","last_clean":"2024-09-22T04:06:49.012301+0000","last_became_active":"2024-09-22T03:46:55.378604+0000","last_became_peered":"2024-09-22T03:46:55.378604+0000","last_unstale":"2024-09-22T04:06:49.012301+0000","last_undegraded":"2024-09-22T04:06:49.012301+0000","last_fullsized":"2024-09-22T04:06:49.012301+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:49.040244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.040244+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2171,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.497675+0000","last_change":"2024-09-22T04:06:49.531965+0000","last_active":"2024-09-22T04:06:49.497675+0000","last_peered":"2024-09-22T04:06:49.497675+0000","last_clean":"2024-09-22T04:06:49.497675+0000","last_became_active":"2024-09-22T03:46:55.370620+0000","last_became_peered":"2024-09-22T03:46:55.370620+0000","last_unstale":"2024-09-22T04:06:49.497675+0000","last_undegraded":"2024-09-22T04:06:49.497675+0000","last_fullsized":"2024-09-22T04:06:49.497675+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:49.531625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.531625+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2171,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.009204+0000","last_change":"2024-09-22T04:06:49.046332+0000","last_active":"2024-09-22T04:06:49.009204+0000","last_peered":"2024-09-22T04:06:49.009204+0000","last_clean":"2024-09-22T04:06:49.009204+0000","last_became_active":"2024-09-22T03:46:55.358980+0000","last_became_peered":"2024-09-22T03:46:55.358980+0000","last_unstale":"2024-09-22T04:06:49.009204+0000","last_undegraded":"2024-09-22T04:06:49.009204+0000","last_fullsized":"2024-09-22T04:06:49.009204+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:49.046140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.046140+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":2171,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.007638+0000","last_change":"2024-09-22T04:06:48.067002+0000","last_active":"2024-09-22T04:06:48.007638+0000","last_peered":"2024-09-22T04:06:48.007638+0000","last_clean":"2024-09-22T04:06:48.007638+0000","last_became_active":"2024-09-22T03:46:55.383021+0000","last_became_peered":"2024-09-22T03:46:55.383021+0000","last_unstale":"2024-09-22T04:06:48.007638+0000","last_undegraded":"2024-09-22T04:06:48.007638+0000","last_fullsized":"2024-09-22T04:06:48.007638+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'2","last_scrub_stamp":"2024-09-22T04:06:48.066796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.066796+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2171,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.509358+0000","last_change":"2024-09-22T04:06:48.548819+0000","last_active":"2024-09-22T04:06:48.509358+0000","last_peered":"2024-09-22T04:06:48.509358+0000","last_clean":"2024-09-22T04:06:48.509358+0000","last_became_active":"2024-09-22T03:46:55.358542+0000","last_became_peered":"2024-09-22T03:46:55.358542+0000","last_unstale":"2024-09-22T04:06:48.509358+0000","last_undegraded":"2024-09-22T04:06:48.509358+0000","last_fullsized":"2024-09-22T04:06:48.509358+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.548529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.548529+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2169,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.007548+0000","last_change":"2024-09-22T04:06:48.041685+0000","last_active":"2024-09-22T04:06:48.007548+0000","last_peered":"2024-09-22T04:06:48.007548+0000","last_clean":"2024-09-22T04:06:48.007548+0000","last_became_active":"2024-09-22T03:46:55.412604+0000","last_became_peered":"2024-09-22T03:46:55.412604+0000","last_unstale":"2024-09-22T04:06:48.007548+0000","last_undegraded":"2024-09-22T04:06:48.007548+0000","last_fullsized":"2024-09-22T04:06:48.007548+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T04:06:48.041493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.041493+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":2172,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:48.511606+0000","last_change":"2024-09-22T04:06:48.561801+0000","last_active":"2024-09-22T04:06:48.511606+0000","last_peered":"2024-09-22T04:06:48.511606+0000","last_clean":"2024-09-22T04:06:48.511606+0000","last_became_active":"2024-09-22T03:46:55.387128+0000","last_became_peered":"2024-09-22T03:46:55.387128+0000","last_unstale":"2024-09-22T04:06:48.511606+0000","last_undegraded":"2024-09-22T04:06:48.511606+0000","last_fullsized":"2024-09-22T04:06:48.511606+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"13'1","last_scrub_stamp":"2024-09-22T04:06:48.561588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:48.561588+0000","last_clean_scrub_stamp":"2024-09-22T03:46:54.283073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":2243,"reported_epoch":733,"state":"active+clean","last_fresh":"2024-09-22T04:06:49.494807+0000","last_change":"2024-09-22T04:06:49.580434+0000","last_active":"2024-09-22T04:06:49.494807+0000","last_peered":"2024-09-22T04:06:49.494807+0000","last_clean":"2024-09-22T04:06:49.494807+0000","last_became_active":"2024-09-22T03:46:53.349788+0000","last_became_peered":"2024-09-22T03:46:53.349788+0000","last_unstale":"2024-09-22T04:06:49.494807+0000","last_undegraded":"2024-09-22T04:06:49.494807+0000","last_fullsized":"2024-09-22T04:06:49.494807+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"12'52","last_scrub_stamp":"2024-09-22T04:06:49.580241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T04:06:49.580241+0000","last_clean_scrub_stamp":"2024-09-22T03:46:52.275666+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":75,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214194618368,"internally_reserved":0,"allocated":16875520,"data_stored":16875520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":214211493888,"available":214194618368,"internally_reserved":0,"allocated":16875520,"data_stored":16875520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53548302336,"internally_reserved":0,"allocated":4571136,"data_stored":4571136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654706878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":4464,"kb_used_data":4464,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52293264,"statfs":{"total":53552873472,"available":53548302336,"internally_reserved":0,"allocated":4571136,"data_stored":4571136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654706878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":3596,"kb_used_data":3596,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52294132,"statfs":{"total":53552873472,"available":53549191168,"internally_reserved":0,"allocated":3682304,"data_stored":3682304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654706878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":3568,"kb_used_data":3568,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52294160,"statfs":{"total":53552873472,"available":53549219840,"internally_reserved":0,"allocated":3653632,"data_stored":3653632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":4852,"kb_used_data":4852,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52292876,"statfs":{"total":53552873472,"available":53547905024,"internally_reserved":0,"allocated":4968448,"data_stored":4968448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53552873472,"available":53548302336,"internally_reserved":0,"allocated":4571136,"data_stored":4571136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53547905024,"internally_reserved":0,"allocated":4968448,"data_stored":4968448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53549219840,"internally_reserved":0,"allocated":3653632,"data_stored":3653632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53549191168,"internally_reserved":0,"allocated":3682304,"data_stored":3682304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53552873472,"available":53548302336,"internally_reserved":0,"allocated":4571136,"data_stored":4571136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":0,"total":53552873472,"available":53547905024,"internally_reserved":0,"allocated":4968448,"data_stored":4968448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":1,"total":53552873472,"available":53549219840,"internally_reserved":0,"allocated":3653632,"data_stored":3653632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":2,"total":53552873472,"available":53549191168,"internally_reserved":0,"allocated":3682304,"data_stored":3682304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":75,"osd":3,"total":53552873472,"available":53548302336,"internally_reserved":0,"allocated":4571136,"data_stored":4571136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T04:07:10.184 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-22T04:07:10.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.289+0000 7fdf06d11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:07:10.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.298+0000 7fdf06d11640 1 Processor -- start 2024-09-22T04:07:10.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.298+0000 7fdf06d11640 1 -- start start 2024-09-22T04:07:10.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.299+0000 7fdf06d11640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf00073720 0x7fdf0013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:07:10.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.299+0000 7fdf06d11640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf000743d0 con 0x7fdf000a1fc0 2024-09-22T04:07:10.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.299+0000 7fdf06d11640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fdf000740c0 con 0x7fdf00073720 2024-09-22T04:07:10.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.300+0000 7fdf04aad640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdf000a1fc0 0x7fdf00073010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37356/0 (socket says 172.21.15.3:37356) 2024-09-22T04:07:10.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.300+0000 7fdf04aad640 1 -- 172.21.15.3:0/807934758 learned_addr learned my addr 172.21.15.3:0/807934758 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-22T04:07:10.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.300+0000 7fdefffff640 1 --2- 172.21.15.3:0/807934758 >> v2:172.21.15.3:3300/0 conn(0x7fdf00073720 0x7fdf0013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:07:10.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.300+0000 7fdefffff640 1 -- 172.21.15.3:0/807934758 >> v1:172.21.15.3:6789/0 conn(0x7fdf000a1fc0 legacy=0x7fdf00073010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:07:10.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.300+0000 7fdefffff640 1 -- 172.21.15.3:0/807934758 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf000a3270 con 0x7fdf00073720 2024-09-22T04:07:10.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.300+0000 7fdefffff640 1 --2- 172.21.15.3:0/807934758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf00073720 0x7fdf0013b160 crc :-1 s=READY pgs=13045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa192b670c7efc71 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:07:10.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.300+0000 7fdeff7fe640 1 -- 172.21.15.3:0/807934758 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdef400e040 con 0x7fdf00073720 2024-09-22T04:07:10.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.300+0000 7fdeff7fe640 1 -- 172.21.15.3:0/807934758 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fdef4007670 con 0x7fdf00073720 2024-09-22T04:07:10.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.300+0000 7fdeff7fe640 1 -- 172.21.15.3:0/807934758 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdef4007910 con 0x7fdf00073720 2024-09-22T04:07:10.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.301+0000 7fdf06d11640 1 -- 172.21.15.3:0/807934758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf00073720 msgr2=0x7fdf0013b160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:07:10.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.301+0000 7fdf06d11640 1 --2- 172.21.15.3:0/807934758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf00073720 0x7fdf0013b160 crc :-1 s=READY pgs=13045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:07:10.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.301+0000 7fdf06d11640 1 -- 172.21.15.3:0/807934758 shutdown_connections 2024-09-22T04:07:10.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.301+0000 7fdf06d11640 1 --2- 172.21.15.3:0/807934758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf00073720 0x7fdf0013b160 unknown :-1 s=CLOSED pgs=13045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:07:10.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.301+0000 7fdf06d11640 1 -- 172.21.15.3:0/807934758 >> 172.21.15.3:0/807934758 conn(0x7fdf0006f0a0 msgr2=0x7fdf000714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:07:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.301+0000 7fdf06d11640 1 -- 172.21.15.3:0/807934758 shutdown_connections 2024-09-22T04:07:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.301+0000 7fdf06d11640 1 -- 172.21.15.3:0/807934758 wait complete. 2024-09-22T04:07:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.301+0000 7fdf06d11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-22T04:07:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.303+0000 7fdf06d11640 1 Processor -- start 2024-09-22T04:07:10.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.303+0000 7fdf06d11640 1 -- start start 2024-09-22T04:07:10.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.305+0000 7fdf06d11640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf00073720 0x7fdf00123830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:07:10.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.305+0000 7fdf06d11640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdf00136c20 con 0x7fdf00073720 2024-09-22T04:07:10.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.305+0000 7fdf04aad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf00073720 0x7fdf00123830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:07:10.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.305+0000 7fdf04aad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf00073720 0x7fdf00123830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36352/0 (socket says 172.21.15.3:36352) 2024-09-22T04:07:10.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.305+0000 7fdf04aad640 1 -- 172.21.15.3:0/3334358538 learned_addr learned my addr 172.21.15.3:0/3334358538 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-22T04:07:10.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.305+0000 7fdf04aad640 1 -- 172.21.15.3:0/3334358538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf00125230 con 0x7fdf00073720 2024-09-22T04:07:10.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.305+0000 7fdf04aad640 1 --2- 172.21.15.3:0/3334358538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf00073720 0x7fdf00123830 crc :-1 s=READY pgs=13046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:07:10.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.306+0000 7fdefd7fa640 1 -- 172.21.15.3:0/3334358538 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdef0005a60 con 0x7fdf00073720 2024-09-22T04:07:10.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.306+0000 7fdefd7fa640 1 -- 172.21.15.3:0/3334358538 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x7fdef0005bc0 con 0x7fdf00073720 2024-09-22T04:07:10.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.306+0000 7fdefd7fa640 1 -- 172.21.15.3:0/3334358538 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (crc 0 0 0) 0x7fdef0005e60 con 0x7fdf00073720 2024-09-22T04:07:10.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.306+0000 7fdf06d11640 1 -- 172.21.15.3:0/3334358538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf001249c0 con 0x7fdf00073720 2024-09-22T04:07:10.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.306+0000 7fdf06d11640 1 -- 172.21.15.3:0/3334358538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf00126050 con 0x7fdf00073720 2024-09-22T04:07:10.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.307+0000 7fdf06d11640 1 -- 172.21.15.3:0/3334358538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf00124bb0 con 0x7fdf00073720 2024-09-22T04:07:10.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.307+0000 7fdefd7fa640 1 -- 172.21.15.3:0/3334358538 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (crc 0 0 0) 0x7fdef000c820 con 0x7fdf00073720 2024-09-22T04:07:10.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.307+0000 7fdefd7fa640 1 --2- 172.21.15.3:0/3334358538 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fdeec03ea80 0x7fdeec040f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-22T04:07:10.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.307+0000 7fdefd7fa640 1 -- 172.21.15.3:0/3334358538 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(733..733 src has 683..733) ==== 3431+0+0 (crc 0 0 0) 0x7fdef004fbc0 con 0x7fdf00073720 2024-09-22T04:07:10.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.308+0000 7fdefffff640 1 --2- 172.21.15.3:0/3334358538 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fdeec03ea80 0x7fdeec040f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-22T04:07:10.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.308+0000 7fdefffff640 1 --2- 172.21.15.3:0/3334358538 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fdeec03ea80 0x7fdeec040f90 crc :-1 s=READY pgs=3883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-22T04:07:10.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.312+0000 7fdefd7fa640 1 -- 172.21.15.3:0/3334358538 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (crc 0 0 0) 0x7fdef000dc60 con 0x7fdf00073720 2024-09-22T04:07:10.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.506+0000 7fdf06d11640 1 -- 172.21.15.3:0/3334358538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fdf00073010 con 0x7fdf00073720 2024-09-22T04:07:10.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.509+0000 7fdefd7fa640 1 -- 172.21.15.3:0/3334358538 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v2)=0 v2) ==== 123+0+0 (crc 0 0 0) 0x7fdef000d660 con 0x7fdf00073720 2024-09-22T04:07:10.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.510+0000 7fdefd7fa640 1 -- 172.21.15.3:0/3334358538 <== mon.0 v2:172.21.15.3:3300/0 8 ==== config(1 keys) ==== 35+0+0 (crc 0 0 0) 0x7fdef000dc60 con 0x7fdf00073720 2024-09-22T04:07:10.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.512+0000 7fdf06d11640 1 -- 172.21.15.3:0/3334358538 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fdeec03ea80 msgr2=0x7fdeec040f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:07:10.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.512+0000 7fdf06d11640 1 --2- 172.21.15.3:0/3334358538 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fdeec03ea80 0x7fdeec040f90 crc :-1 s=READY pgs=3883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:07:10.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.512+0000 7fdf06d11640 1 -- 172.21.15.3:0/3334358538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf00073720 msgr2=0x7fdf00123830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-22T04:07:10.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.512+0000 7fdf06d11640 1 --2- 172.21.15.3:0/3334358538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf00073720 0x7fdf00123830 crc :-1 s=READY pgs=13046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:07:10.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.512+0000 7fdf06d11640 1 -- 172.21.15.3:0/3334358538 shutdown_connections 2024-09-22T04:07:10.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.512+0000 7fdf06d11640 1 --2- 172.21.15.3:0/3334358538 >> [v2:172.21.15.123:6800/551784226,v1:172.21.15.123:6801/551784226] conn(0x7fdeec03ea80 0x7fdeec040f90 unknown :-1 s=CLOSED pgs=3883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:07:10.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.512+0000 7fdf06d11640 1 --2- 172.21.15.3:0/3334358538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf00073720 0x7fdf00123830 unknown :-1 s=CLOSED pgs=13046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-22T04:07:10.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.512+0000 7fdf06d11640 1 -- 172.21.15.3:0/3334358538 >> 172.21.15.3:0/3334358538 conn(0x7fdf0006f0a0 msgr2=0x7fdf000a5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-22T04:07:10.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.512+0000 7fdf06d11640 1 -- 172.21.15.3:0/3334358538 shutdown_connections 2024-09-22T04:07:10.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-22T04:07:10.512+0000 7fdf06d11640 1 -- 172.21.15.3:0/3334358538 wait complete. 2024-09-22T04:07:10.526 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-22T04:07:10.526 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-22T04:07:10.527 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-22T04:07:10.527 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T04:07:10.607 INFO:tasks.ceph.osd.0:Stopped 2024-09-22T04:07:10.608 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-22T04:07:10.608 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T04:07:10.686 INFO:tasks.ceph.osd.1:Stopped 2024-09-22T04:07:10.687 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-22T04:07:10.687 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T04:07:10.753 INFO:tasks.ceph.osd.2:Stopped 2024-09-22T04:07:10.753 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-22T04:07:10.754 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T04:07:10.826 INFO:tasks.ceph.osd.3:Stopped 2024-09-22T04:07:10.826 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-22T04:07:10.827 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-22T04:07:10.827 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T04:07:10.856 INFO:tasks.ceph.mgr.x:Stopped 2024-09-22T04:07:10.856 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-22T04:07:10.856 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-22T04:07:10.856 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T04:07:10.871 INFO:tasks.ceph.mon.a:Stopped 2024-09-22T04:07:10.871 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-22T04:07:10.871 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-09-22T04:07:10.955 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-22T04:07:10.955 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-22T04:07:11.173 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-22T04:07:11.174 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-22T04:07:11.297 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-22T04:07:11.297 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-22T04:07:11.407 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-22T04:07:11.407 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-22T04:07:11.496 INFO:tasks.ceph:Archiving mon data... 2024-09-22T04:07:11.497 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi/7914369/data/mon.a.tgz 2024-09-22T04:07:11.497 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-09-22T04:07:11.513 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.3dqfmM100X 2024-09-22T04:07:11.514 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.3dqfmM100X 2024-09-22T04:07:11.663 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.3dqfmM100X 2024-09-22T04:07:11.747 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.3dqfmM100X is 309KB 2024-09-22T04:07:11.841 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.3dqfmM100X 2024-09-22T04:07:11.857 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-22T04:07:11.857 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-22T04:07:11.898 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-22T04:07:11.978 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-22T04:07:11.993 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-22T04:07:11.994 INFO:tasks.ceph:Archiving crash dumps... 2024-09-22T04:07:11.994 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi/7914369/remote/smithi003/crash 2024-09-22T04:07:11.995 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-22T04:07:12.043 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi/7914369/remote/smithi123/crash 2024-09-22T04:07:12.043 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-22T04:07:12.102 INFO:tasks.ceph:Compressing logs... 2024-09-22T04:07:12.102 DEBUG:teuthology.orchestra.run.smithi003:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-22T04:07:12.105 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-22T04:07:12.127 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-22T04:07:12.127 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94338.log 2024-09-22T04:07:12.129 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93034.log 2024-09-22T04:07:12.129 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.94338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93227.log 2024-09-22T04:07:12.129 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.93034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94078.log 2024-09-22T04:07:12.129 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94338.log.gz 2024-09-22T04:07:12.129 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.93227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96388.log 2024-09-22T04:07:12.130 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.94078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92701.log 2024-09-22T04:07:12.130 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96868.log 2024-09-22T04:07:12.134 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27998.log 2024-09-22T04:07:12.134 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27918.log 2024-09-22T04:07:12.134 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-22T04:07:12.134 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.27998.log: /var/log/ceph/ceph-client.admin.27918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27869.log 2024-09-22T04:07:12.134 INFO:teuthology.orchestra.run.smithi123.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.27998.log.gz 2024-09-22T04:07:12.134 INFO:teuthology.orchestra.run.smithi123.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.27918.log.gz 2024-09-22T04:07:12.134 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-22T04:07:12.135 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.27869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27958.log 2024-09-22T04:07:12.135 INFO:teuthology.orchestra.run.smithi123.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.27869.log.gz 2024-09-22T04:07:12.135 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-22T04:07:12.135 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.27958.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.27958.log.gz 2024-09-22T04:07:12.179 INFO:teuthology.orchestra.run.smithi003.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.96388.log.gz 2024-09-22T04:07:12.179 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94078.log.gz 2024-09-22T04:07:12.179 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.92701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31055.log 2024-09-22T04:07:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95727.log 2024-09-22T04:07:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96868.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.93034.log.gz 2024-09-22T04:07:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93976.log 2024-09-22T04:07:12.180 INFO:teuthology.orchestra.run.smithi003.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.96868.log.gz 2024-09-22T04:07:12.180 INFO:teuthology.orchestra.run.smithi003.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.93227.log.gz 2024-09-22T04:07:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94686.log 2024-09-22T04:07:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95727.log: /var/log/ceph/ceph-client.admin.93976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28204.log 2024-09-22T04:07:12.180 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95727.log.gz 2024-09-22T04:07:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.94686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95853.log 2024-09-22T04:07:12.180 INFO:teuthology.orchestra.run.smithi003.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.93976.log.gz 2024-09-22T04:07:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.28204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93208.log 2024-09-22T04:07:12.180 INFO:teuthology.orchestra.run.smithi003.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.28204.log.gz 2024-09-22T04:07:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95898.log 2024-09-22T04:07:12.181 INFO:teuthology.orchestra.run.smithi003.stderr:gzip/var/log/ceph/ceph-client.admin.93208.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.93937.log 2024-09-22T04:07:12.181 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95853.log.gz 2024-09-22T04:07:12.181 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95898.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-22T04:07:12.181 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.93937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29229.log 2024-09-22T04:07:12.181 INFO:teuthology.orchestra.run.smithi003.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.95898.log.gz 2024-09-22T04:07:12.181 INFO:teuthology.orchestra.run.smithi003.stderr: 83.7% -- replaced with /var/log/ceph/ceph-client.admin.93937.log.gz 2024-09-22T04:07:12.181 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.1.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.93208.log.gz 2024-09-22T04:07:12.181 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30110.log 2024-09-22T04:07:12.181 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93189.log 2024-09-22T04:07:12.181 INFO:teuthology.orchestra.run.smithi003.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.29229.log.gz 2024-09-22T04:07:12.181 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94408.log 2024-09-22T04:07:12.181 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.93189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97036.log 2024-09-22T04:07:12.181 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.94408.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.93189.log.gz 2024-09-22T04:07:12.181 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95748.log 2024-09-22T04:07:12.182 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.97036.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.97036.log.gz 2024-09-22T04:07:12.182 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95790.log 2024-09-22T04:07:12.182 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95748.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.94408.log.gz 2024-09-22T04:07:12.182 INFO:teuthology.orchestra.run.smithi003.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.94686.log.gz 2024-09-22T04:07:12.182 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94161.log 2024-09-22T04:07:12.182 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95748.log.gz 2024-09-22T04:07:12.182 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29641.log 2024-09-22T04:07:12.182 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.95790.log.gz 2024-09-22T04:07:12.182 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.96130.log 2024-09-22T04:07:12.182 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29641.log: /var/log/ceph/ceph-client.admin.94161.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27416.log 2024-09-22T04:07:12.182 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96130.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.96130.log.gz 2024-09-22T04:07:12.182 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94161.log.gz 2024-09-22T04:07:12.182 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96106.log 2024-09-22T04:07:12.183 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29227.log 2024-09-22T04:07:12.183 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph.tmp-client.admin.27416.log: 68.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27416.log.gz 2024-09-22T04:07:12.183 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28892.log 2024-09-22T04:07:12.183 INFO:teuthology.orchestra.run.smithi003.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.96106.log.gz 2024-09-22T04:07:12.183 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29227.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.29227.log.gz 2024-09-22T04:07:12.183 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95942.log 2024-09-22T04:07:12.183 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.28892.log: 95.0% -- replaced with /var/log/ceph/ceph-client.admin.92701.log.gz 2024-09-22T04:07:12.183 INFO:teuthology.orchestra.run.smithi003.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.28892.log.gz 2024-09-22T04:07:12.183 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29060.log 2024-09-22T04:07:12.183 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94932.log 2024-09-22T04:07:12.183 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29060.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95942.log.gz 2024-09-22T04:07:12.183 INFO:teuthology.orchestra.run.smithi003.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.29641.log.gz 2024-09-22T04:07:12.183 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95685.log 2024-09-22T04:07:12.183 INFO:teuthology.orchestra.run.smithi003.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.29060.log.gz 2024-09-22T04:07:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31017.log 2024-09-22T04:07:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.94932.log: gzip/var/log/ceph/ceph-client.admin.95685.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.95420.log 2024-09-22T04:07:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31017.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95685.log.gz 2024-09-22T04:07:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31036.log 2024-09-22T04:07:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94302.log 2024-09-22T04:07:12.184 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31017.log.gz 91.6% -- replaced with /var/log/ceph/ceph-client.admin.94932.log.gz 2024-09-22T04:07:12.184 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-22T04:07:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95420.log: /var/log/ceph/ceph-client.admin.31036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96345.log 2024-09-22T04:07:12.184 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7%/var/log/ceph/ceph-client.admin.94302.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-22T04:07:12.184 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31036.log.gz 2024-09-22T04:07:12.184 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95420.log.gz 2024-09-22T04:07:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96742.log 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96345.log.gz 2024-09-22T04:07:12.184 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-22T04:07:12.185 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94302.log.gz 2024-09-22T04:07:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28791.log 2024-09-22T04:07:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95252.log 2024-09-22T04:07:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.28791.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.96742.log.gz 2024-09-22T04:07:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94340.log 2024-09-22T04:07:12.185 INFO:teuthology.orchestra.run.smithi003.stderr: 86.2% 93.4% -- replaced with /var/log/ceph/ceph-client.admin.30110.log.gz 2024-09-22T04:07:12.185 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28791.log.gz 2024-09-22T04:07:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95874.log 2024-09-22T04:07:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95252.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-22T04:07:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.94340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95452.log 2024-09-22T04:07:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5/var/log/ceph/ceph-client.admin.95874.log: --verbose -- /var/log/ceph/ceph-client.admin.93294.log 2024-09-22T04:07:12.185 INFO:teuthology.orchestra.run.smithi003.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.94340.log.gz/var/log/ceph/ceph-client.admin.95452.log: 2024-09-22T04:07:12.185 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95874.log.gz 2024-09-22T04:07:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95659.log 2024-09-22T04:07:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:gzip/var/log/ceph/ceph-client.admin.93294.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.96658.log 2024-09-22T04:07:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29379.log 2024-09-22T04:07:12.186 INFO:teuthology.orchestra.run.smithi003.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.95659.log.gz 2024-09-22T04:07:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94025.log 2024-09-22T04:07:12.186 INFO:teuthology.orchestra.run.smithi003.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.95252.log.gz 91.0% -- replaced with /var/log/ceph/ceph-client.admin.93294.log.gz 2024-09-22T04:07:12.186 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-22T04:07:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29379.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.96658.log.gz 2024-09-22T04:07:12.186 INFO:teuthology.orchestra.run.smithi003.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.29379.log.gz 2024-09-22T04:07:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96437.log 2024-09-22T04:07:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.94025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95963.log 2024-09-22T04:07:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28707.log 2024-09-22T04:07:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95897.log 2024-09-22T04:07:12.186 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% 86.1% -- replaced with /var/log/ceph/ceph-client.admin.96437.log.gz 2024-09-22T04:07:12.187 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94025.log.gz 2024-09-22T04:07:12.187 INFO:teuthology.orchestra.run.smithi003.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.95963.log.gz/var/log/ceph/ceph-client.admin.28707.log: 2024-09-22T04:07:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97120.log 2024-09-22T04:07:12.187 INFO:teuthology.orchestra.run.smithi003.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.28707.log.gz 2024-09-22T04:07:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96700.log 2024-09-22T04:07:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.97120.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95897.log.gz 2024-09-22T04:07:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30979.log 2024-09-22T04:07:12.187 INFO:teuthology.orchestra.run.smithi003.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.97120.log.gz 2024-09-22T04:07:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28581.log 2024-09-22T04:07:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30979.log: 85.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94163.log 2024-09-22T04:07:12.187 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96700.log.gz 2024-09-22T04:07:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.28581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28283.log 2024-09-22T04:07:12.187 INFO:teuthology.orchestra.run.smithi003.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.28581.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93939.log 2024-09-22T04:07:12.187 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-22T04:07:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.94163.log: /var/log/ceph/ceph-client.admin.28283.log: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.28283.log.gz 2024-09-22T04:07:12.188 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94163.log.gz 2024-09-22T04:07:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30887.log 2024-09-22T04:07:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94391.log 2024-09-22T04:07:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.93939.log: /var/log/ceph/ceph-client.admin.30887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95990.log 2024-09-22T04:07:12.188 INFO:teuthology.orchestra.run.smithi003.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.95452.log.gz 2024-09-22T04:07:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.94391.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.93939.log.gzgzip -5 85.9% -- replaced with /var/log/ceph/ceph-client.admin.94391.log.gz 2024-09-22T04:07:12.188 INFO:teuthology.orchestra.run.smithi003.stderr: --verbose -- /var/log/ceph/ceph-client.admin.93680.log 2024-09-22T04:07:12.188 INFO:teuthology.orchestra.run.smithi003.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.30887.log.gz 2024-09-22T04:07:12.188 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-22T04:07:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96910.log 2024-09-22T04:07:12.188 INFO:teuthology.orchestra.run.smithi003.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.95990.log.gz 2024-09-22T04:07:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95989.log 2024-09-22T04:07:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.93680.log: /var/log/ceph/ceph-client.admin.96910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94079.log 2024-09-22T04:07:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95989.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.96910.log.gz 2024-09-22T04:07:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28749.log 2024-09-22T04:07:12.189 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95989.log.gz/var/log/ceph/ceph-client.admin.94079.log: 2024-09-22T04:07:12.189 INFO:teuthology.orchestra.run.smithi003.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.93680.log.gz 2024-09-22T04:07:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96324.log 2024-09-22T04:07:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5/var/log/ceph/ceph-client.admin.28749.log: --verbose 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94079.log.gz 2024-09-22T04:07:12.189 INFO:teuthology.orchestra.run.smithi003.stderr: -- /var/log/ceph/ceph-client.admin.93168.log 2024-09-22T04:07:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96324.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.28749.log.gz 2024-09-22T04:07:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94230.log 2024-09-22T04:07:12.189 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96324.log.gz 2024-09-22T04:07:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.93168.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-22T04:07:12.189 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.94230.log: gzip -5 --verbose 87.6% -- /var/log/ceph/ceph-client.admin.28439.log 2024-09-22T04:07:12.189 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93168.log.gz 2024-09-22T04:07:12.190 INFO:teuthology.orchestra.run.smithi003.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.94230.log.gz 2024-09-22T04:07:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94303.log 94.1% -- replaced with /var/log/ceph/ceph-client.admin.30979.log.gz 2024-09-22T04:07:12.190 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-22T04:07:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.28439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30940.log 2024-09-22T04:07:12.190 INFO:teuthology.orchestra.run.smithi003.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.28439.log.gz 2024-09-22T04:07:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.94303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96574.log 2024-09-22T04:07:12.190 INFO:teuthology.orchestra.run.smithi003.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.94303.log.gz 2024-09-22T04:07:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96080.log 2024-09-22T04:07:12.190 INFO:teuthology.orchestra.run.smithi003.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.30940.log.gz 2024-09-22T04:07:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94076.log 2024-09-22T04:07:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96080.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.96574.log.gz 2024-09-22T04:07:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96058.log 2024-09-22T04:07:12.190 INFO:teuthology.orchestra.run.smithi003.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.96080.log.gz 2024-09-22T04:07:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.94076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96034.log 2024-09-22T04:07:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96058.log: gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.admin.93975.log 2024-09-22T04:07:12.191 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94076.log.gz 2024-09-22T04:07:12.191 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96058.log.gz 2024-09-22T04:07:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96303.log 2024-09-22T04:07:12.191 INFO:teuthology.orchestra.run.smithi003.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.96034.log.gz 2024-09-22T04:07:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.93975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92997.log 2024-09-22T04:07:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96303.log: gzip -5 --verbose 90.6% -- replaced with /var/log/ceph/ceph-client.admin.93975.log.gz 2024-09-22T04:07:12.191 INFO:teuthology.orchestra.run.smithi003.stderr: -- /var/log/ceph/ceph-client.admin.92813.log 2024-09-22T04:07:12.191 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96303.log.gz 2024-09-22T04:07:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.92997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94339.log 2024-09-22T04:07:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.92813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93149.log 2024-09-22T04:07:12.191 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.94339.log: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95624.log 2024-09-22T04:07:12.191 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92997.log.gz 2024-09-22T04:07:12.192 INFO:teuthology.orchestra.run.smithi003.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.94339.log.gz 2024-09-22T04:07:12.192 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.93149.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93149.log.gz 2024-09-22T04:07:12.192 INFO:teuthology.orchestra.run.smithi003.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.92813.log.gz 2024-09-22T04:07:12.192 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29061.log 2024-09-22T04:07:12.192 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94027.log 2024-09-22T04:07:12.192 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95624.log: /var/log/ceph/ceph-client.admin.29061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96282.log 2024-09-22T04:07:12.192 INFO:teuthology.orchestra.run.smithi003.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.95624.log.gz 2024-09-22T04:07:12.192 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.94027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96412.log 2024-09-22T04:07:12.192 INFO:teuthology.orchestra.run.smithi003.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.29061.log.gz 2024-09-22T04:07:12.192 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29421.log 2024-09-22T04:07:12.192 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-22T04:07:12.192 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94027.log.gz 2024-09-22T04:07:12.192 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96282.log.gz 2024-09-22T04:07:12.192 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96412.log: /var/log/ceph/ceph-client.admin.29421.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.29421.log.gz 2024-09-22T04:07:12.193 INFO:teuthology.orchestra.run.smithi003.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.96412.log.gz 2024-09-22T04:07:12.193 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96994.log 2024-09-22T04:07:12.193 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94229.log 2024-09-22T04:07:12.193 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28467.log 2024-09-22T04:07:12.193 INFO:teuthology.orchestra.run.smithi003.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.96994.log.gz/var/log/ceph/ceph-client.admin.94229.log: 2024-09-22T04:07:12.193 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94265.log 2024-09-22T04:07:12.193 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97160.log 2024-09-22T04:07:12.193 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3%/var/log/ceph/ceph-client.admin.28467.log: -- replaced with /var/log/ceph/ceph-client.admin.94229.log.gz 2024-09-22T04:07:12.193 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.94265.log: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.28467.log.gz 2024-09-22T04:07:12.193 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94077.log 2024-09-22T04:07:12.193 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94265.log.gz 2024-09-22T04:07:12.193 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28893.log 2024-09-22T04:07:12.193 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.97160.log: /var/log/ceph/ceph-client.admin.94077.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.97160.log.gz 2024-09-22T04:07:12.194 INFO:teuthology.orchestra.run.smithi003.stderr: 89.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.94077.log.gz 2024-09-22T04:07:12.194 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.95706.log 2024-09-22T04:07:12.194 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-22T04:07:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95811.log 2024-09-22T04:07:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.28893.log: /var/log/ceph/ceph-client.admin.95706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94472.log 2024-09-22T04:07:12.194 INFO:teuthology.orchestra.run.smithi003.stderr: 86.4% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95706.log.gz 2024-09-22T04:07:12.194 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28893.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93345.log 2024-09-22T04:07:12.194 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-22T04:07:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95811.log: /var/log/ceph/ceph-client.admin.94472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96156.log 2024-09-22T04:07:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.93345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96261.log 2024-09-22T04:07:12.194 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95811.log.gz 2024-09-22T04:07:12.194 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95769.log 2024-09-22T04:07:12.194 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96156.log.gz 2024-09-22T04:07:12.195 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96261.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96261.log.gz 2024-09-22T04:07:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95039.log 2024-09-22T04:07:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96461.log 2024-09-22T04:07:12.212 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95769.log.gz 2024-09-22T04:07:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95039.log: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.94472.log.gz 2024-09-22T04:07:12.212 INFO:teuthology.orchestra.run.smithi003.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.93345.log.gz 2024-09-22T04:07:12.212 INFO:teuthology.orchestra.run.smithi003.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.95039.log.gz 2024-09-22T04:07:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96784.log 2024-09-22T04:07:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29337.log 2024-09-22T04:07:12.212 INFO:teuthology.orchestra.run.smithi003.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.96461.log.gz 2024-09-22T04:07:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95832.log 2024-09-22T04:07:12.212 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29337.log: gzip -5 --verbose -- 85.7% /var/log/ceph/ceph-client.admin.29226.log 2024-09-22T04:07:12.213 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96784.log.gz 2024-09-22T04:07:12.213 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95832.log: 85.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28665.log 2024-09-22T04:07:12.213 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29337.log.gz 2024-09-22T04:07:12.213 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.95832.log.gz 2024-09-22T04:07:12.213 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97078.log 2024-09-22T04:07:12.213 INFO:teuthology.orchestra.run.smithi003.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29226.log.gz 2024-09-22T04:07:12.213 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.28665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29059.log 2024-09-22T04:07:12.213 INFO:teuthology.orchestra.run.smithi003.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.28665.log.gz 2024-09-22T04:07:12.213 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.97078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28525.log 2024-09-22T04:07:12.213 INFO:teuthology.orchestra.run.smithi003.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.97078.log.gz 2024-09-22T04:07:12.213 INFO:teuthology.orchestra.run.smithi003.stderr:gzip/var/log/ceph/ceph-client.admin.29059.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.93789.log 2024-09-22T04:07:12.213 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.28525.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.29059.log.gz 2024-09-22T04:07:12.213 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94080.log 2024-09-22T04:07:12.214 INFO:teuthology.orchestra.run.smithi003.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.28525.log.gz 2024-09-22T04:07:12.214 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.93789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93938.log 2024-09-22T04:07:12.214 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5/var/log/ceph/ceph-client.admin.94080.log: --verbose -- /var/log/ceph/ceph-client.admin.96367.log 2024-09-22T04:07:12.214 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.93938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94026.log 2024-09-22T04:07:12.214 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94080.log.gz 2024-09-22T04:07:12.214 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96367.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94266.log 2024-09-22T04:07:12.214 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93938.log.gz 2024-09-22T04:07:12.214 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7%/var/log/ceph/ceph-client.admin.94026.log: -- replaced with /var/log/ceph/ceph-client.admin.96367.log.gz 2024-09-22T04:07:12.214 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95598.log 2024-09-22T04:07:12.214 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.94266.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.94026.log.gz 2024-09-22T04:07:12.214 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28614.log 2024-09-22T04:07:12.214 INFO:teuthology.orchestra.run.smithi003.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.94266.log.gz 2024-09-22T04:07:12.214 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96952.log 2024-09-22T04:07:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.95598.log: /var/log/ceph/ceph-client.admin.28614.log: 85.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94734.log 2024-09-22T04:07:12.215 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95598.log.gz 2024-09-22T04:07:12.215 INFO:teuthology.orchestra.run.smithi003.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.28614.log.gz 2024-09-22T04:07:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28891.log 2024-09-22T04:07:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96952.log: /var/log/ceph/ceph-client.admin.94734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93918.log 2024-09-22T04:07:12.215 INFO:teuthology.orchestra.run.smithi003.stderr: 85.7%/var/log/ceph/ceph-client.admin.28891.log: -- replaced with /var/log/ceph/ceph-client.admin.96952.log.gz 2024-09-22T04:07:12.215 INFO:teuthology.orchestra.run.smithi003.stderr: 86.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96219.log 2024-09-22T04:07:12.215 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28891.log.gz 2024-09-22T04:07:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94162.log 2024-09-22T04:07:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.93918.log: /var/log/ceph/ceph-client.admin.96219.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96219.log.gz 2024-09-22T04:07:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94455.log 2024-09-22T04:07:12.215 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.94162.log: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.93918.log.gz 2024-09-22T04:07:12.215 INFO:teuthology.orchestra.run.smithi003.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.94734.log.gz 2024-09-22T04:07:12.216 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96826.log 89.9% -- replaced with /var/log/ceph/ceph-client.admin.94162.log.gz 2024-09-22T04:07:12.216 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-22T04:07:12.216 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.94455.log: gzip -5 --verbose -- 85.9% -- replaced with /var/log/ceph/ceph-client.admin.94455.log.gz 2024-09-22T04:07:12.216 INFO:teuthology.orchestra.run.smithi003.stderr: /var/log/ceph/ceph-client.admin.96240.log 2024-09-22T04:07:12.216 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93974.log 2024-09-22T04:07:12.216 INFO:teuthology.orchestra.run.smithi003.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.96826.log.gz 2024-09-22T04:07:12.216 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96616.log 2024-09-22T04:07:12.216 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96240.log.gz 2024-09-22T04:07:12.216 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28890.log 2024-09-22T04:07:12.216 INFO:teuthology.orchestra.run.smithi003.stderr: 92.8%/var/log/ceph/ceph-client.admin.93974.log: -- replaced with /var/log/ceph/ceph-client.admin.93789.log.gz 2024-09-22T04:07:12.216 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29169.log 2024-09-22T04:07:12.216 INFO:teuthology.orchestra.run.smithi003.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.96616.log.gz 2024-09-22T04:07:12.216 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.28890.log: gzip 88.4% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.93974.log.gz -- 2024-09-22T04:07:12.216 INFO:teuthology.orchestra.run.smithi003.stderr: /var/log/ceph/ceph-client.admin.29058.log 2024-09-22T04:07:12.217 INFO:teuthology.orchestra.run.smithi003.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.28890.log.gz 2024-09-22T04:07:12.217 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92775.log 2024-09-22T04:07:12.217 INFO:teuthology.orchestra.run.smithi003.stderr: 86.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30921.log 2024-09-22T04:07:12.217 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29058.log: -- replaced with /var/log/ceph/ceph-client.admin.29169.log.gz 2024-09-22T04:07:12.217 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.92775.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.29058.log.gzgzip 2024-09-22T04:07:12.217 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.96198.log 2024-09-22T04:07:12.217 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.30921.log: -- /var/log/ceph/ceph-client.admin.96177.log 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92775.log.gz 2024-09-22T04:07:12.217 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-22T04:07:12.217 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96198.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96198.log.gz 2024-09-22T04:07:12.217 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.96177.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96177.log.gz 2024-09-22T04:07:12.217 INFO:teuthology.orchestra.run.smithi003.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.30921.log.gz 2024-09-22T04:07:12.346 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.3.log: 92.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-22T04:07:12.585 INFO:teuthology.orchestra.run.smithi003.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.31055.log.gz 2024-09-22T04:07:13.760 INFO:teuthology.orchestra.run.smithi003.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-22T04:07:20.110 INFO:teuthology.orchestra.run.smithi123.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-22T04:07:20.207 INFO:teuthology.orchestra.run.smithi123.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-22T04:07:20.209 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-22T04:07:20.209 INFO:teuthology.orchestra.run.smithi123.stderr:real 0m8.090s 2024-09-22T04:07:20.209 INFO:teuthology.orchestra.run.smithi123.stderr:user 0m15.677s 2024-09-22T04:07:20.209 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m0.583s 2024-09-22T04:07:21.165 INFO:teuthology.orchestra.run.smithi003.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-22T04:07:22.465 INFO:teuthology.orchestra.run.smithi003.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-22T04:07:22.467 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-22T04:07:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:real 0m10.349s 2024-09-22T04:07:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:user 0m20.869s 2024-09-22T04:07:22.467 INFO:teuthology.orchestra.run.smithi003.stderr:sys 0m0.785s 2024-09-22T04:07:22.467 INFO:tasks.ceph:Archiving logs... 2024-09-22T04:07:22.468 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi/7914369/remote/smithi003/log 2024-09-22T04:07:22.469 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-22T04:07:25.743 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi/7914369/remote/smithi123/log 2024-09-22T04:07:25.744 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-22T04:07:28.351 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-22T04:07:28.358 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-22T04:07:28.359 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-22T04:07:28.361 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-22T04:07:28.458 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-22T04:07:28.466 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-22T04:07:28.466 DEBUG:teuthology.orchestra.run.smithi003:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T04:07:28.469 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T04:07:28.484 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-09-22T04:07:28.486 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-09-22T04:07:28.489 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-22T04:07:28.489 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-09-22T04:07:28.490 INFO:teuthology.orchestra.run.smithi003.stdout:^* hv01.front.sepia.ceph.com 3 6 377 3 -514us[ -525us] +/- 50ms 2024-09-22T04:07:28.490 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 72 +974us[ +952us] +/- 57ms 2024-09-22T04:07:28.490 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 68 -486us[ -508us] +/- 63ms 2024-09-22T04:07:28.490 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T04:07:28.491 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-22T04:07:28.492 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-09-22T04:07:28.492 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv01.front.sepia.ceph.com 3 6 377 3 -477us[ -470us] +/- 50ms 2024-09-22T04:07:28.492 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 +972us[ +972us] +/- 58ms 2024-09-22T04:07:28.492 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 -481us[ -474us] +/- 64ms 2024-09-22T04:07:28.492 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-22T04:07:28.493 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-22T04:07:28.504 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-22T04:07:28.504 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-22T04:07:28.515 DEBUG:teuthology.orchestra.run.smithi003:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-22T04:07:28.601 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-22T04:07:28.668 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-22T04:07:28.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T04:07:28.698 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-22T04:07:28.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T04:07:28.728 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-22T04:07:28.737 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-22T04:07:28.745 INFO:teuthology.task.internal:Duration was 1660.992512 seconds 2024-09-22T04:07:28.745 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-22T04:07:28.802 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-22T04:07:28.803 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-22T04:07:28.805 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-22T04:07:28.846 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-22T04:07:28.847 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-22T04:07:29.223 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-22T04:07:29.223 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi003.front.sepia.ceph.com 2024-09-22T04:07:29.223 DEBUG:teuthology.orchestra.run.smithi003:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-22T04:07:29.287 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-09-22T04:07:29.288 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-22T04:07:29.319 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-22T04:07:29.319 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-22T04:07:29.330 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-22T04:07:29.387 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-22T04:07:29.387 DEBUG:teuthology.orchestra.run.smithi003:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-22T04:07:29.400 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-22T04:07:29.527 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-22T04:07:29.543 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-22T04:07:29.543 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-22T04:07:29.569 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-22T04:07:29.594 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-22T04:07:29.603 DEBUG:teuthology.orchestra.run.smithi003:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-22T04:07:29.612 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-22T04:07:29.634 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = core 2024-09-22T04:07:29.657 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-09-22T04:07:29.679 DEBUG:teuthology.orchestra.run.smithi003:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-22T04:07:29.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T04:07:29.703 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-22T04:07:29.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T04:07:29.733 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-22T04:07:29.743 INFO:teuthology.task.internal:Transferring archived files... 2024-09-22T04:07:29.743 DEBUG:teuthology.misc:Transferring archived files from smithi003:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi/7914369/remote/smithi003 2024-09-22T04:07:29.743 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-22T04:07:29.778 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi/7914369/remote/smithi123 2024-09-22T04:07:29.778 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-22T04:07:29.810 INFO:teuthology.task.internal:Removing archive directory... 2024-09-22T04:07:29.810 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-22T04:07:29.815 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-22T04:07:29.860 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-22T04:07:29.870 INFO:teuthology.task.internal:Not uploading archives. 2024-09-22T04:07:29.870 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-22T04:07:29.878 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-22T04:07:29.879 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-22T04:07:29.881 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-22T04:07:29.894 INFO:teuthology.orchestra.run.smithi003.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 22 04:07 /home/ubuntu/cephtest 2024-09-22T04:07:29.914 INFO:teuthology.orchestra.run.smithi123.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 22 04:07 /home/ubuntu/cephtest 2024-09-22T04:07:29.915 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-22T04:07:29.924 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-22T04:07:29.945 INFO:teuthology.run:Summary data: description: crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore tasks/rados_api_tests} duration: 1660.992511510849 flavor: crimson owner: scheduled_teuthology@teuthology success: true 2024-09-22T04:07:29.945 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-22T04:07:30.018 INFO:teuthology.run:pass